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ABSTRACT 



This guide provides ways in which school board members can 
lead their school systems to higher levels of student achievement. The text 
attempts to help board members by equipping them with comprehensive 
information about the various components of student achievement. Part 1 
analyzes the purposes of education and defines student achievement, and 
standards and achievement. Part 2 presents a four-part approach to developing 
an action plan: vision setting, establishing a successful learning 
environment, exercising accountability, and creating vital connections with 
the community. Suggestions for developing, implementing, and overseeing a 
student -achievement agenda and the vital process of public engagement are 
also discussed in this section. Part 3 then provides a guide to testing and 
other indicators of student achievement and school quality, and offers 
guidance in interpreting test scores and understanding such concepts as 
selection bias, statistical significance, and educational productivity. The 
last section discusses other indicators of achievement and school quality and 
shows how by using measures, such as levels of teacher education or per-pupil 
expenditures, school boards can assess progress and thus design programs to 
reflect their communities' unique values. Two appendices provide student 
achievement resources and opportunity to learn indicators. (RUM) 
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Foreword 



we prepare to enter the 21st century, raising student achievement in our nation's schools 



as become the principal domestic policy issue facing communities across America. At 
national, state, and community levels, this vital theme has become the focus of unprecedented 
debate and energy. Across the United States, local school board members and state school 
boards associations are seizing the initiative to raise student achievement. They are frequently 
joined by parents, the business community, civic leaders, and the media, all of whom have an 
important stake in the education of our nation's young people. 

The National School Boards Association (NSBA) has made the educational achievement of all 
children its chief priority. Over the next year, information, especially designed by NSBA to sup- 
port the leadership of local school boards on student achievement issues, will be featured at our 
annual conference, on our Web site, and through association publications. NSBA is committed 
to providing local school board members and state school boards associations with relevant 
and comprehensive information to enhance local school boards' effectiveness in setting poli- 
cies to raise student achievement in their communities. 

This primer, which is intended to serve school boards of every size and in every locale (and 
which is being made available without charge to Federation and National Affiliate school dis- 
tricts), is a substantively rich guide. It provides: (1) the content knowledge that local school 
boards will need to make the best decisions about raising student achievement; (2) the strate- 
gies that school boards can use to govern more effectively; and (3) a guide to other key infor- 
mation sources. In short, this primer is designed to help school board members use the power 
of their office to lead their school systems to higher levels of student achievement. 

We hope this primer will be useful to you and that it will support our shared goal to enhance 
school board effectiveness in raising student achievement in local communities across our great 
nation. 




Sincerely, 





President 



Executive Director 



Introduction 



R aising student achievement is the most important challenge facing local school boards 
today. While the challenge is clear, the depth and scope of the issues and the dynamics 
involved in creating an action plan to raise achievement are complex. The goal of this primer 
is to encourage local school board members to use the power of their office to lead their school 
systems to higher levels of student achievement. This publication aims to help school board 
members become more effective educational policy leaders in two ways: by equipping them 
with comprehensive information about the various components of student achievement and by 
offering suggested strategies for reaching their targets. 

Part One analyzes the purposes of education, which, in turn, will help answer the essential first 
question: "What do you want student achievement to mean in your school district?" Since 
todays debate on student achievement is often tied to the core issue of raising the academic per- 
formance of students, as measured by specific standardized tests, this part also provides a broad 
overview of standards and the current standards movement. 



Part Two of the primer deals directly with the role of the local school board. 

School board members will want to use the substantive content in this publica- 
tion to design a blueprint for action. This section presents a four-part approach 
to developing an action plan: vision setting, establishing a successful learning 
environment, exercising accountability, and creating vital connections with the 
community. Part Two also contains "The Power of the Question," which suggests 
basic, strategic questions school board members may ask as they begin to devel- 
op, implement, and oversee a student achievement agenda. While these questions have a 
strong standards-based reform component to them, they also describe other dimensions of 
student achievement. Finally, Part Two deals with the vital process of public engagement. It 
describes steps school boards can take to encourage the community to join with them to 
define a vision and goals and to strengthen programs and support for student achievement 
activities. 




Part Three, a guide to testing and other indicators of student achievement, examines the vari- 
ety of standardized achievement tests which many school districts across the country currently 
use to measure academic achievement. This section offers guidance in interpreting test scores 
and understanding such concepts as selection bias, statistical significance, and educational pro- 
ductivity. It also should help school board members create the important connections between 
test results (and what they do or do not reveal) and the policy, budgetary, and program deci- 
sions school boards will make based upon those results. 



Because tests are not the sole indicators of how a student, school, or school system performs, 
Part Four discusses other indicators of achievement and school quality. Using measures such 
as the level of teacher education or per-pupil expenditures, school boards can assess progress 
and design programs to reflect their communities' unique demographic profiles and values. 



No single document can provide all the information available on student achievement. 
However, Appendix A lists key government agencies and organizations that can supply a wealth 
of material, including useful studies and innovative practices, to school board members. Taken 
together, these additional resources can offer local school board members a plethora of ideas 
to raise achievement levels in their school systems. 




vu 
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As an addendum to Part Three, Appendix B discusses several key out-of-school variables that 
can influence student achievement, such as poverty, mobility, and demographic trends. These 
factors, which relate to both education and the quality of students lives, should be weighed 
carefully as school boards develop their achievement plans. 

Finally, NSBA urges school board members to keep one overarching question in mind as they 
read this primer: "How can my board and I use this information to create a plan that success- 
fully raises the achievement levels of all students in our district?" 



Part One: 

Setting the Context 




Defining Student Achievement 

E ducation deals with the "good life," observed Aristotle. Thus, it follows that, inasmuch as 
people disagree over what constitutes the good life, there will be a corresponding disagree- 
ment over the purposes of education. In light of these differences, before embarking on the task 
of developing an action plan to raise achievement, school boards must first define student 
achievement in ways that are meaningful to the individual communities they represent. Board 
members must ask: "What is the vision that we hold in common for individual students and 
for the entire school system when we say students are achieving?" 

While the current debate over educational achievement often narrowly focuses on students' 
ability to perform well on certain standardized tests, student achievement can be more broad- 
ly defined to include an array of educational goals: 

• Academic attainment; 

• Job skills and preparation; 

• Citizenship (e.g., volunteerism, voting); 

• Appreciation of the arts; and 

• Development of character and values (e.g., integrity, patriotism, work ethic). 

These goals, which relate both to individual quality of life and to the broader public interest in 
education, might well become part of a school board's definition of student achievement and 
might be included in a district's plans and programs. A brief look at several purposes of edu- 
cation that this nation has historically valued can serve as a starting point for school board dis- 
cussion. 



Civic Purposes 

Thomas Jefferson saw one purpose of education as that of protecting us from our government, 
and, quite likely, he would be appalled at the low voter turnouts and civic apathy characteristic 
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Thomas Jefferson 
saw one purpose of 
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our government, 
and, quite likely, he 
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apathy characteristic 
of modem-day 
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of modem-day America. Jefferson might well attribute these shortcomings to 
the failure of schools to instill in students an understanding of their proper role 
in a democratic society. 

In authoritarian and totalitarian societies, people are told what to think, and 
critical thinking can be dangerous. But, in a democracy, people must leam to 
think for themselves, carefully evaluating the information flowing to them — 
whether it be the claims of an automobile maker, the views of candidates com- 
peting for public office, or conflicting assertions concerning the effects of gas 
emissions on the environment. 



In discussing the purposes of education, school boards may want to consider 
the ways in which their school district ensures that students leam to think criti- 
cally at a time when young people are inundated with more information from 

— — : more sources than was ever before possible. Board members may also want to 

look beyond state or district-mandated citizenship tests to determine how well their school sys- 
tem is preparing students to accept the responsibilities of citizenship in a democratic, pluralis- 
tic society. 



Economic Purposes 

One of the key purposes of education has been to enhance the economic well-being of both 
the individual and the nation as a whole. These two goals are interrelated, since America's eco- 
nomic health depends upon the purchasing power of its citizens and its ability to compete in 
world markets. 

The school's role in helping people up the economic ladder of this society, recognized since the 
Civil War, is even stronger today. While education does not guarantee a specific wage, it does 
provide the opportunity to earn a higher income, as several indices demonstrate. For example, 
according to the first International Adult Literacy Survey, Americans reading at the highest two 
levels are more likely to be in the fourth and fifth quintiles of income, while very few individ- 
uals (a mere 3.6 percent) in the two highest income categories read at the lowest level. Another 
survey found that completion of each educational level, from high school diploma through 
doctorate, virtually doubles the average income of its holders. 

Education not only affects an individual's chances, but it also has an economic impact on soci- 
ety as a whole. In fact, many view schooling as "The Great Sorting Machine" because it affects 
the distribution of wealth in this country. That role is likely to increase as we continue to move 
from an industrial society, which offered an abundance of low-skilled but not always low-pay- 
ing jobs, to an information society, where knowledge is a key factor for employment. Those 
with little or inadequate schooling are likely to be left by the wayside. And, while current unem- 
ployment rates are low, due in part to the increase of jobs in the service sector, many of these 
positions pay low wages and offer only part-time employment. The average work week for 
today's retail salespeople, for example, is just 28 hours. 

The quality of schools and the students they produce also influence our nation's ability to com- 
pete globally, although that link is less clear than many imagine. "A Nation At Risk," published 
in 1983, contended that "If only to keep and improve on the slim competitive edge we still 
retain in world markets, we must dedicate ourselves to the reform of our educational system for 
the benefit of all." That world view became very popular in the late 1980s, when the economy 



slid into recession and some version of the "lousy-schools-are-producing-a-lousy- 
workforce-and-that-is-killing-us-in-the-global-market" theme could be heard in 
many quarters. Others, however, saw from the beginning that the link between 
the quality of America's schools and a weak economy is tenuous, at best. In 
Popular Education and Its Discontents (1989), the distinguished education histori- 
an Lawrence Cremin made this observation: 

American economic competitiveness with Japan and other nations is to a 
erable degree a function of monetary, trade, and industrial policy, and of d< 
made by the President and Congress, the Federal Reserve Board, and the 
departments of the Treasury and Commerce and Labor. 

The accuracy of Cremin's comments became evident in late 1993, when the economy came 
roaring back even while critics claimed that schools had not improved. In The Atlantic Monthly, 
journalist Peter Schrag declared that it was "embarrassing" to read the thesis of "A Nation At 
Risk" today. That point was further underscored with the recent collapse of Asian economies. 
The entire Southeast Asia Region had to ask international agencies for bailouts — even though 
Asian students continued to score high marks on international math and science tests. 

While the quality of our schools may be overstated as an influence on the current national econ- 
omy, at the local level, student achievement can have a very real influence on the business deci- 
sions that are made and the economic well-being of specific communities. When faced with 
expansion or relocation decisions, firms do take into account the quality of schools as one fac- 
tor. For example, when the German car manufacturer BMW decided to build an automotive 
plant in South Carolina, it weighed its options carefully, basing its site selection, in part, on the 
quality of local schools. 

Education cannot guarantee that all people will achieve their employment or income goals. 
Flowever, in 21st century America, large segments of the population will likely never have ade- 
quate resources without a solid education. Therefore, school boards must determine whether 
their schools are presenting a challenging curriculum that allows all students to obtain the 
knowledge and skills they need to participate fully in the economic opportunities this society, 
and the world, offer. Local boards will want to make certain that their policies are not inad- 
vertently tracking certain segments of their school populations into jobs with little future. 




Aesthetic Purposes 

Beyond its civic and economic purposes, education also serves an important aesthetic purpose 
that contributes to the quality of life and general well-being of society. Mathematician Israel 
Sheffler defined education this way: "The formation of habits of judgment and the develop- 
ment of character, the elevation of standards, the facilitation of understanding, the develop- 
ment of taste and discrimination, the stimulation of curiosity and wondering, the fostering of 
style and a sense of beauty, the growth of a thirst for new ideas and visions of the yet unknown." 
While some of these qualities would assist a person in a quest for a high-paying job, many of 
them have no immediate utilitarian function. They are qualities we might consider good out- 
comes in and of themselves. School board members might want to keep Shefflers definition 
in mind as they discuss the many dimensions of achievement. 
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One purpose of edu- 
cation is to enable 
people to have rich 
intellectual, social, 
and emotional lives 
off the job, as well 
as to be productive 
on it. 



Although the American worker is the most productive in the world, many people 
worry about the tendency of Americans to watch hours upon hours of mindless 
television and to engage in activities that benefit neither family nor community. 
One purpose of education is to enable people to have rich intellectual, social, and 
emotional lives off the job, as well as to be productive on it. Again, school boards 
might want to consider whether they are producing students who have gained the 
habit of critical judgment and the desire for lifelong learning. 



Finally, school board members must re-examine the values and characteristics 
which make their communities unique. As they discuss the individual, local, and 
societal purposes of education, local school board members may want to con- 
sider these questions: Is our school system fulfilling its role? What does our school system have 
to do — and what do our students need to leam — to fulfill our vision of a school district where 
all students are achieving at their highest potential? Answers to these questions will help in 
developing a plan of action to raise achievement, however one may define it. 



Standards and Achievement 

As Americans focus their attention on achievement in the nation's schools, there is growing con- 
cern about the accountability of schools and school districts. Concerns about meeting stan- 
dards are at the center of this debate. 

In the present context, "standards" refer to both subject content and student performance lev- 
els. Content standards mean the knowledge or skills that students need to possess at a partic- 
ular grade level or in a particular course. Performance standards refer to the level of proficien- 
cy or mastery of knowledge that students must acquire — as measured by a test. That is, how 
high do we set the bar? 

Virtually every state (Iowa is an exception) has initiated standards-based reform which focuses 
on state-prescribed standards. While these standards will influence local curriculum by pre- 
scribing minimum competencies which students must achieve, they do not tell a school district 
how to reach those goals, nor do they set any limits on what students can achieve. 

Beyond this, standards raise certain alignment issues which must be addressed. If students are 
to achieve the standards set by the state or local school system, those standards and the tests 
that measure how well the standards are being met must be closely aligned. Using a test that 
does not accurately measure whether or not students are achieving the standards that have been 
set does not provide useful information and may, in fact, lead to misguided decisions for 
"improvement." Likewise, both standards and tests must be aligned with the resources (such 
as textbooks and curriculum) that are made available to students. 



The Standards Movement: Historical Background 

Calls for school reform were heard in earnest just after World War II. These calls were largely 
for changes in what was perceived to be an outdated curriculum. Students were studying 
Newton's laws when they should have been learning about subatomic particles and memoriz- 
ing phyla when they needed to know about evolution and ecology. More recently, the idea has 
been put forth that American students do not achieve more than they do because they are not 
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held to high standards. In their review of the National Assessment of Educational Progress 
(NEAP); Willard Wirtz and Archie Lapointe wrote: 



By the early 1970s, the national sense developed that educational quality was dete- 
riorating rapidly and dangerously. It was not entirely clear how far this went, how 
general it was, or what measure of fault was the schools'.. .So it was decided. ..in one 
State after another... to move in directly on the schools, not with funds, but with 
"standards." ( Measuring the Quality of Education, 1982). 



Historically, standards have provided a means of recording magnitude without 
imposing value judgments. Thus, at the National Institute of Standards and 
Technology, carefully calibrated platinum rods are used to define standards of 
length, but they are silent on whether a particular length is good or bad. 
Standards have been used in this way in education as well. But, since the 1980s, 
education has adopted a different definition of standards, a definition which 
contains a sense of requirement, excellence, and attainment. That is, standards 
today frequently refer not only to what students do and do not know, which was 
the role of the National Assessment initially, but also prescribe what they ought 
to know. Specific standards of performance are established and students are held 
accountable. The reality of applying this definition of standards has been a bit 
more complicated than the rhetoric, however. 



Students were study- 
ing Newton's laws 
when they should 
have been learning 
about subatomic 
particles and memo- 
rizing phyla when 
they needed to know 
about evolution and 
ecology. 



Outcomes-Based Education 

The first round of standard-setting activities generally went under the name of Outcomes- 
Based Education (OBE). OBE was initially a purely cognitive program that would define what 
children should know and be able to do and measure how well they performed on these objec- 
tives. All students would be presented with a challenging curriculum. 

The OBE movement soon confronted two problems, however. First, as states began to define 
"outcomes," many slipped out of the purely cognitive realm and into, on occasion, the affec- 
tive. They inquired about students 7 "feelings" and, in some instances, prescribed how students 
should feel. They delved into self-esteem. In so doing, they moved into areas that some peo- 
ple thought were improper intrusions of the school (and, therefore, the state) into a realm 
properly left to the family. 

There was another objection to OBE, although not quite so inflammatory. OBE advocates con- 
tended that "all children can learn." They would teach all children the same challenging cur- 
riculum. Critics, however, mindful of the idea of the bell curve, contended that the only way 
to teach all children the same thing was not to teach any of them very much. They thus saw 
OBE as a disaster in the making for able students. If your goal is to have all students slam-dunk 
a basketball, went the argument, you would have to lower the basket. 



Subject-Matter Standards 

Those twin critiques derailed the OBE movement, but the desire for higher standards was left 
intact. Various academic institutions and professional groups, following the lead of the 
National Council of Teachers of Mathematics (NCTM), promulgated content standards. Some, 
as with the NCTM mathematics standards and the science standards developed by the National 



2 3 mm copy available 



Academy of Sciences, were generally met with wide acceptance. Many critics derided other stan- 
dards, though — such as language arts and history standards — as being meaningless or merely 
politically correct. Board members might want to review these standards and the various com- 
ments expressed about them. 

Whatever the reactions to standards emanating from professional organizations, they were mild 
compared with resistance to the federal government's proposed national standards and a "vol- 
untary" series of national tests. Those proposals ran into the same anti-government buzz saw 
that OBE had encountered, but resistance now resided, not with the secular and religious Right, 
but in the statehouses of the nation. In 1996, the governors of the country declared themselves 
all in favor of standards but did not support a federal effort to develop them. 

Although it has been more than 15 years since the Wirtz and Lapointe report, it is still too early 
to judge the impact of the standards movement. There are, currently, many efforts at the state 
level to develop both standards and tests. For the reasons discussed below, school board mem- 
bers, in setting their own standards, should recognize that state standards represent an impor- 
tant factor and are presently undergoing changes that will impact local school systems. 



New State Standards 




In 1989, President George Bush and the nation's 50 governors held an education summit 
which resulted in the establishment of six (and later eight) national goals in education. 
Among these goals, those pertaining to student achievement — including the goal of our 
nation becoming first in the world in mathematics and science by the year 2000 — became the 
most prominent on the list. 




Over the years, the National Education Goals Panel (NEGP) has spearheaded the 
work of encouraging states to implement high standards. That group, which is 
closely aligned with the National Governors' Association, includes governors, 
state legislators, and members of Congress and the executive branch (including 
the secretary of education). 



In 1996, a second summit, made up of business leaders and governors, was con- 
vened to identify specific strategies and to introduce newer state leaders to the standards 
process. From this summit, an entity called "Achieve" was created to provide technical assis- 
tance to the states and to help states benchmark their standards and assessments. It is one thing 
for a state to adopt a set of standards, but how do those standards compare with those of other 
states? 



Within this climate of increasing national (not federal) collaboration, many states have been 
working to establish higher standards. This effort has received funding through a variety of 
sources, including the federal Goals 2000 program. The current state-based standards move- 
ment is a powerful and interconnecting force that can be expected to influence local school 
boards' student achievement efforts for some time to come. States are moving rapidly to ensure 
that school districts meet at least minimum expectations of achievement in reading and math- 
ematics. Thus, school board members are urged to consider the merits and the politics of the 
standards in their state, as well as those in their own local school systems, and their relation- 
ship to student achievement goals. (Appendix A of this primer lists a variety of information 
sources on standards.) 
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Part Two: 

The Role of the Local School Board 




he public expects the nation's schools to provide a learning environment conducive to stu- 



dent achievement at the highest levels. It is the fundamental role of local school boards to 
provide the leadership necessary to create that learning environment To provide such leader- 
ship, local school boards must first establish student achievement as their highest priority. 

Historically, local school boards, as lay governors of the school system, believed that their role 
was not to substitute their own views on matters of pedagogy for those of professional educa- 
tors. Rather, they perceived their role to be supportive in nature, approving the budget, placing 
their stamp on legal documents, dealing with constituents, receiving reports, campaigning for 
bond issues, and providing "cover" on politically sensitive issues. 

Although all of these school board functions are legitimate and should continue, the challenges 
of raising student achievement in the 21st century suggest a very clear and overarching gover- 
nance role for local school boards in setting education policy. It is a role that does not cross 
into the implementation of educational content or pedagogy, but that provides leadership to 
school systems as they establish and achieve high levels of performance among their students. 

While many school districts today are immersed in standard setting, reading-improvement 
strategies, or other specific components of student achievement, many have not yet addressed 
the issue in its entirety. The purpose of this section is to help local school boards raise their spe- 
cific goals and strategies to a new level of thinking by focusing on the "whole" of student 
achievement for all children. 



The Four Pillars of the Local School Board Role 



To advance student achievement, local school boards can structure their work into four broad 
and interrelated categories: 

• Vision setting; 

• Establishing a successful learning environment; 
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• Exercising accountability for results; and 

• Using advocacy to build support. 

Each is a necessary component of successful school board leadership. School boards should 
review their current activities in each area, consider what more could be done, and evaluate how 
these four basic functions can be more effectively linked to each other to produce a well- 
designed action plan. 



Vision Setting for Student Achievement 



Local school boards 
should play the cen- 
tral role in driving 
and guiding the 
process to establish 
|a vision of education 
for their school 
systems. 



Local school boards should play the central role in driving and guiding the 
process to establish a vision of education for their school systems. Indeed, as rep- 
resentatives of the community and governors of the school system, school boards 
are the best catalysts for stimulating the dialogue, consensus, and actions that can 
shape a truly dynamic and responsive student achievement plan. Local school 
boards should play the central role in driving and guiding the process to estab- 
lish a vision of education for their school systems. 



Successful vision setting will involve several steps, such as establishing school dis- 
trict goals, developing indicators of success, and drawing comparisons to the cur- 
, rent system. The process also must bring all parties — parents, the business com- 
munity, the media, and the community at large, together with administrators, teachers (includ- 
ing unions), and the school board — into a common dialogue about the future of their school 
system. 



Establishing a Student Achievement Environment 

To be successful, the vision must be supported by an operational plan that: 

• Provides the necessary programs and resources; 

• Promotes the understanding, commitment, and accountability of all parties; and 

• Devotes the time and energy that is needed at all levels within the system. 

The superintendent, as chief executive officer of the school district, establishes the operating 
environment. Compatibility between the superintendent and the district's specific student 
achievement vision should be at the forefront of his or her hiring and evaluation. At the same 
time, the school board must provide sufficient resources and flexibility to create an environ- 
ment which allows the superintendent to focus on bringing the vision to fruition. 

Beyond providing the financial resources, the school board must take the lead in creating an 
environment for student learning in other key ways. It should make student achievement a sig- 
nificant part of its regular meering agenda, pursue it in board committee work, and advocate it 
throughout the community. A successful school board must operate as a coherent team, with 
clear strategies and goals in mind. 



Accountability for Student Achievement 

In establishing accountability measures, the school board and staff must determine the way 
success and failure will be assessed, the content and timing of reports, the system of rewards and 



consequences, and the method for reporting goals, results, and other information to individual 
parents and to the public at large. 

Most important, the school system must hold itself accountable for the implementation of the 
student achievement plan and, ultimately, for student success. An accountability process must 
detail the contributions expected of the superintendent, teachers, principals, and the board 
itself. It should also ensure that the vision, plan, and specific expectations for student achieve- 
ment are communicated to staff members as early as possible and made a part of their own for- 
mal evaluation process. 

Finally, a student achievement plan must also hold students accountable for their own per- 
formance. Students must know how good is good enough, how success will be measured, and 
what positive and negative consequences (such as special recognition for excellence or limits on 
participation in extracurricular activities for failure) the school system has in place. The respon- 
sibilities of parents as partners in the educational process also need to be constructively and 
informatively communicated. 



Advocacy for Student Achievement 

Meeting student achievement goals will require the understanding, involvement, 
and support of the local community and of state and federal policymakers. The 
local school board must be a vocal advocate for its student achievement vision 
to each of these audiences. 

Because more than 70 percent of households do not have children in public 
school, the school district needs to communicate its vision and student achieve- 
ment goals very clearly to taxpayers and the community as a whole to ensure 
their support. Without this information, the public's perspective and support 
can be skewed by the anecdotes and advocacy of others who themselves may not 
have the entire picture. By contrast, comprehensive information from the school 
district is likely both to increase support for school budgets and to improve the 
chances of attracting the involvement of businesses and others in school activi- 
ties. 

State and federal lawmakers and other policymakers also need to know the 
school district's goals and how they can help support them. State legislators, 
state board of education members, the state superintendent, and the governor, 
as well as members of the U.S. Congress, have the power to shape the direction of school sys- 
tems. Therefore, these policymakers should be encouraged to focus their energy on supporting 
the school system's vision and plan rather than working at cross purposes to it or in ignorance 
of it. 
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The Power of the Question 



Focusing the school system on student achievement issues and strategies is one of the most 
important functions of a local school board, and board members often can accomplish this task 
simply by asking the right questions. The following sections suggest several questions school 
boards can pose to the superintendent and staff in such key areas as standard setting, testing, 
and resource alignment. 
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Setting High Standards: Measuring Academic Results 



As previously noted, content standards identify what children need to know and do, and per- 
formance standards identify the level of proficiency that students are expected to reach. 
Virtually all states set these standards for school systems and use a variety of mechanisms to 
obtain compliance, including mandating standardized testing and requiring students to take 
certain courses to earn a diploma. While local school districts must adhere to minimum state 
standards, they are not precluded from requiring students to perform at even higher levels than 
those set by the state. 

It is important that parents, the school board, and staff, at appropriate levels, 
know what the standards are and what the tests measure. The school district 
should ensure that tests accurately measure standards in order to avoid expensive 
and potentially disruptive program changes based on invalid test results. Here are 
some questions local school boards should ask about standards and tests: 

1. Do we want the state's standards to be the "all" or the "core" of our school system's stan- 
dards? How much better do we want our students to perform than the state requires? What 
else do we want them to know and do? 

2. How well do our teachers know our school district's standards and the level of instruction 
that is needed to reach them? How clearly are standards being communicated to parents 
and the community? Do parents know ways they can help their child reach the standards? 

3. Which tests do we use to measure student achievement? What are they designed to tell us? 
Can the results be misinterpreted and, if so, how? 

4. Do the tests we use emphasize skills in memorization, analysis, application, communica- 
tion, etc., in a balance that reflects the school district's concept of student achievement? 

5. In addition to any state test that is required, are there any other tests that we want to use? 

6. How vigorous are the tests and standards that we use? For example, while our students may 
perform well on the state's test for language skills, do we know how well the state as a whole 
performs on a national test such as the National Assessment of Educational Progress 
(NAEP)? If the children of our state perform significantly better on the state test than on 
NAEP, do we need to re-examine our state test and standards? 

7. How does our school district want to benchmark test results? For example, do we want to 
compare ourselves with the state as a whole, with neighboring school districts, with school 
districts that resemble our demographic and socioeconomic profile, with our performance 
in previous years, with our vision or goals, or with the percentage of our students that move 
up or down from one category of proficiency to another in a given year? 

8. What performance standards and expectations do we set for children with special needs? 
Do we provide special support so that they can maximize their level of performance? 

9. What kinds of rewards and remedies are there for students and staff for exceptionally good 
or bad results? 

10. How well do parents and community members understand test results? Are specific areas 
of a student's strengths and weaknesses clearly identified and defined? 
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Other Indicators of Student Achievement 



In addition to performance on standardized tests, are there other indicators of student achieve- 
ment that our school district should use? If we use other indicators, how do we determine 
whether our efforts are successful? To a great extent, the selection of these indicators will define 
the character and values that distinguish local communities from each other and are at the heart 
of local governance of the schools. 

The following questions (along with the detailed outline provided in Part Four) demonstrate 
the broad range of indicators by which school districts can measure goals and success beyond 
performance on standardized academic tests: 

1. What percentage of our students graduate? What percentage graduate with the state's high- 
est academic diploma? 

2. What percentage of our students plan to enroll in two-year and four-year colleges immedi- 
ately after graduation? 

3. What does the business community think of the competency of our recent graduates? How 
can we find out if we don't know? 

4. What percentage of our students take Advanced Placement courses? What percentage take 
three or four years of mathematics and science (rather than business mathematics and gen- 
eral science)? What percentage take three or more years of foreign language? 

5. Are greater numbers of women, minorities, or other subgroups who have been historically 
underrepresented in higher-level mathematics and science courses now taking these cours- 
es? 

6. What percentage of our students are engaged in positive (e.g., volunteer) activities in the 
community? 

7. Have we adequately reduced violence and drug abuse? 

8. Finally, are our students prepared to meet the challenges of the 21st century— not just in an 
academic sense, but also in terms of the technological, organizational, and sociocultural 
demands of the next century? 



Resource Alignment for Student Achievement 



As previously discussed, tests must accurately reflect the standards that are set, 
or a school system will not know how well its students are meeting those stan- 
dards and will not have a valid basis for making program changes. 



But even when standards and tests are properly aligned with each other, the sys- 
tem's resources and programs must also be adequately designed to support the 
standards. Not even top school districts are likely to meet all the tested stan- 
dards if their programs highlight skills and knowledge which differ significant- 
ly from those that the standards emphasize. 
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Resource alignment issues such as these are addressed in the following broad questions and in 

the ensuing discussion, which examines the specific areas of teacher qualifications, curriculum 

development, and textbook selection: 

1. Does our school district have the range of programs in place and the quality of program- 
ming needed to reach our student achievement goals? (Students cannot take Advanced 
Placement courses, for example, if they are not offered.) 

2. Do we have the quality and adequacy of special support programs (teen pregnancy preven- 
tion programs, for example) needed to help students stay in school? 

3. Does our school district have current information about innovations and best practices 
across the state or country in such areas as whole-school reform or the use of technology in 
raising student achievement? 

4. If a particular school is not adequately meeting our school system's standards, what do we 
do about it? Are there any formal processes in place to help that school improve? 

5. Do we set high expectations for all children? In supporting children to reach those expec- 
tations, do we accommodate different learning needs, as well as the special needs of indi- 
vidual students and our culturally diverse student body? 



Aligning Teachers with High Standards 

The quality of the teaching force is at the core of student achievement. In a paper prepared for 
the National Goals Panel, "Teaching For High Standards: What Policy Makers Need To Know 
And Be Able To Do," Columbia University education professor Linda Darling-Hammond 
reviewed a number of studies which suggest that teacher qualifications (as measured by educa- 
tion, experience, and licensing exam scores) exert an important influence on student achieve- 
ment. However, several of these studies employed data aggregated to the district 
level, making it difficult to establish the precise relationship between teacher 
expertise and achievement at the individual level. 

Darling-Hammond also cites data from 1990-91 to demonstrate that too many 
new teachers are hired with emergency licenses (25 percent), and, especially at 
the high school level, too many do not even have a college minor in their field 
of instruction (23 percent). More than half of high school students taking phys- 
ical science are taught by out-of-field teachers (56 percent), as are nearly two- 
fifths of those taking life science (39 percent), and roughly a quarter of those tak- 
ing mathematics (27 percent). Further, the least qualified teachers tend to teach 
in schools with the highest poverty/minority enrollments. She also raises con- 
cerns over the adequacy of schools of education in preparing new teachers and 
the professional development opportunities available to current teachers. 

While these broad observations will apply to school districts differently, they 
provide a useful context for asking the questions set forth below: 
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1. By subject area and grade level, what percentage of our teachers are qualified (through cer- 
tification, license, or college major) to teach in their field of instruction? What is the vari- 
ation in this percentage by school building, and why? What percentage of our new teachers 
hold substandard or provisional licenses? 
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2. Do our teachers possess a variety of effective pedagogical skills in their sub- 
ject area so that they have the skills to accommodate different student learn- 
ing needs? 

3. How do our teachers' qualifications compare with those of teachers in simi- 
lar local school systems or in school systems that we would like to emulate? 



4. Do we assign our new teachers or our best teachers to the most challenging students and 
to the most challenging schools? 

5. How closely are professional development opportunities and compensation rewards tied jf j 
to the teacher's subject area? 





6. Do teachers know the standards expected in our local school system? 

7. Can teachers access the information they need? Do they know how to use new technolo- 
gies in an educationally effective manner? 



8. To raise the quality of teaching in our school system, what actions do the school system and 
the school board need to take? How long will these actions take? How much will they cost? 
How will our collective bargaining contract have to change? 

9. Beyond workshops, what kind of professional development opportunities are available to 
our teachers (e.g., professional networks, mentoring programs, in-school group activities)? 
Do we have adequate resources for professional development? Do professional develop- 
ment activities focus on alignment issues and best practices? 

10. Do our teachers have adequate time and resources to design curriculum and plan lessons? 



11. What mechanisms, such as classroom observation, do we have to determine whether teach- 
ers are teaching to the standard that is held for students? (That standard might include 
teaching broader concepts, such as real-world applications in mathematics and science.) 



Aligning Curriculum with High Standards 

Aligning curriculum with the school district's academic standards and other indicators of suc- 
cess is critical to attain the system's vision for student achievement. School boards can ask a 
variety of questions to ensure that the school system's curriculum and curriculum development 
are effective: 

1. Has our school district adjusted its curriculum to reflect new state standards, as well as the 
school system's own standards? 

2. Are there areas in which state standards are impinging upon our local flexibility to deter- 
mine curriculum? 

3. Do our school district and the state have adequate numbers of competent personnel to 
assist local curriculum development? 
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4. What support systems exist to align classroom teaching and curriculum with curriculum 
frameworks? Do our teachers have access to subject knowledge and models for effective 
teaching? 

5. Do we know what our teachers need to know to teach to standards? How well are states 
and state universities preparing new teachers for changes in school curriculum? 



Aligning Textbooks with High Standards 

Textbooks and other instructional materials should be the "meat and potatoes" for bringing 
standards into the classroom. Yet textbooks in many subject areas often are not substantive, do 
not explain their subjects thoroughly and frequently do not articulate well with related subjects 
and other grade levels of instruction. 
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Many observers have offered numerous criticisms of textbooks, particularly 
those in mathematics and science. In a paper prepared for the National 
Education Goals Panel, George D. Nelson of the American Association for the 
Advancement of Science contends that science and mathematics textbooks (and 
curriculum generally) are a "mile wide and an inch deep," do not adequately 
treat specific subject matter, and too frequently emphasize memorization over 
understanding. 



Other observers, such as education writer Harriet Tyson, also raise the concern 
that publishers "dumbed down" their texts and went overboard covering too 
broad a range of subject matter in order to be "marketable" to as diverse a range 
of school districts as possible. Certain textbooks may be selected without ade- 
quate review or because they come with a teacher instructional guide, even 
■ though other, higher quality textbooks, may be available. These observations 
may not apply equally to all school systems, but they do suggest a series of questions that can 
help determine whether or not textbooks and the selection process support increased student 
achievement: 



1. Are our school districts textbooks aligned with the subject matter and the level of per- 
forance that we want our children to achieve? 

2. Do our textbooks provide appropriate treatment of specific subject matter to promote stu- 
dent understanding and acquisition of knowledge? Do they incorporate and expedite best 
practices? 

3. What is the formal process for textbook selection in our school district? 

4. Do the persons responsible for textbook selection have adequate training and time to do 
the job? 

5. To what extent are better textbooks rejected because of the attraction of bonuses (such as 
videos and instructional guides) that accompany the sale of other textbooks? 

6. What external sources do we use to help review textbooks? Do these external reviews eval- 
uate a textbook based on the quality of content? Or, are these reviews based on such fac- 
tors as the number of times a subject is mentioned (regardless of context or the meaning 
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conveyed) in order to demonstrate that the text is covering terms used in the state's stan- 
dards and curriculum frameworks? 



The Mega Questions 

Through the "Power of the Question," local school boards can learn the key issues to raise and 

can provide the leadership necessary to elevate student achievement. 

Now, with these substantive questions in mind, there are a few broader questions that can pro- 
vide a context for evaluating a school system's student achievement effort and help each board 

initiate a plan of action: 

1. How does our school system define student achievement? 

2. Does our school system have a vision and a plan for raising the achievement of all children? 

3. Are our student achievement goals, plans, and progress reports set forth in clear and quan- 
tifiable terms and broadly disseminated within the school system to parents and to the gen- 
eral public? 

4. Are our teachers, administrators, and other staff committed to, and held accountable for, 
achieving the goals and standards of our school system? 

5. How do we involve parents, the business community, and other members of the public in 
the development and implementation of, and the accountability for, our student achieve- 
ment goals? 

6. What next steps can our school board take to lead the effort to raise student achievement 
in our school system? 

7. Finally, are our students prepared to meet the challenges of the 21st century — not just in an 
academic sense, but also in terms of the technological, organizational, and sociocultural 
demands of the next century? 



Public Engagement and Community Outreach 



It is at the community level that the American public can have the greatest 
impact on education. And it is at their own community level where citizens 
most want the schools to succeed. To that end, local businesses frequently sup- 
port schools by contributing equipment and supplies and by offering work- 
study or other incentive programs to keep students in school. Civic and reli- 
gious groups also work with schools to help develop character, provide support 
programs, and create positive influences that undergird the education of the 
whole child. Similarly, news media that understand the vision and direction of 
the school system can play a very important role in building and attracting com- 
munity support. Most important, many school boards recognize the vital role 
parents play in the education of their children and have encouraged them to 
take an active part in schools and in developing learning partnerships. 
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At its most basic level, public engagement can mean a project to inform the public about the 
school system or a public relations effort to create a positive climate for school initiatives. It 
also may involve dialogue among diverse community groups to listen to the public's percep- 
tion of the school system and its goals for the future. But taken to its fullest, public engage- 
ment means convening the community for the purpose of developing a common vision from 
which the school system can develop a student achievement plan — and gain com- 
munity support for its implementation. 

Public engagement can be built around a single issue. Ultimately, however, pub- 
lic engagement should be built around the school system's entire student achieve- 
ment program for all students. (One good example of this more comprehensive 
approach is the Cincinnati Public Schools' Strategic Plan, described on NSBA's 
Web site, www.nsba.org.) 




To be effective, the school board should build its public engagement program with the four pil- 
lars of the school board role in mind (vision setting, the provision of a high-quality learning 
environment, proper accountability, and community advocacy). The board needs to plan the 
various steps carefully, because process is vitally important in getting results. Before engaging 
the community, school system representatives at all levels (the school board, administrators, 
and teachers) should understand the process and develop a consensus about the most effective 
approach. For example, the use of small focus groups may be a better way to begin engaging 
the public than a mass meeting, where people may not fully understand what is expected of 
them. 



Taking the steps out of sequence may hinder results. For example, simply calling the commu- 
nity into an initial vision-setting meeting may result in a low turnout or a meeting which draws 
only well-known activists but no new blood or broadened community perspective. If school 
system representatives outnumber the community, they may inhibit dialogue or turn the meet- 
ing into a "school system tells all" event. They may spend too much time and use too much 
jargon defending the current system rather than listening. 

Boards may want to consider using an experienced outside facilitator to develop and manage 
their public engagement process. The state school boards association may have a public engage- 
ment service and should be a good source of ideas. 

To help evaluate their current public engagement activities, local school boards can ask the fol- 
lowing questions: 

1. Is the community involved in telling us its viewpoint about our school system, its needs, 
and expectations? Does the process really involve communication with a broad range of 
constituents? 

2. Is the community involved in establishing the vision and goals for the school district? 

3. Is the community informed of the system's vision, success, failures, and plans for improve- 
ment? 

4. Do our staff and community understand the vision and goals of the school system? 

5. Do we report test results and other information relating to our indicators of success to par- 
ents, the business community, the media, etc.? Do our reports promote understanding, or 
do they lead to inaccurate conclusions? How do we know? 



6. When we report to the public, do we present our goals and results in dear 
quantifiable terms (e.g., in three years we want 95 percent of our fifth-grade 
students to meet state standards for reading)? 

7. What efforts do we make to actively communicate with special groups and 
target audiences to address the six engagement activities outlined above? 

School board engagement also means involvement at the state level and with other entities 
that influence or can be a resource in raising local student achievement. Local school boards 
may want to ask several questions to determine whether the school system is "ahead of the 
curve" in developing student achievement polides and innovations that incorporate and sup- 
port best practices: 





1. To what extent is our school system initiating or responding to proposals at the state level 
to change standards or assessments, textbook selection, course requirements, or profes- 
sional qualifications? Do we want to? 

2. What kinds of state technical assistance or grant opportunities are available to help support 
our student achievement plan? 

3. To what extent do we use the standards set by national professional associations (e.g., the 
National Council of Teachers of Mathematics)? Do we want to? 

4. Does our school district have adequate outreach to various governmental and quasi-gov- 
emmental agendes, as well as to education associations, to keep abreast of current devel- 
opments, trends, and practices in student achievement? 

5. Do our school board members and staff have adequate opportunities to sharpen their stu- 
dent achievement strategies through conferences and publications? 

Creating and implementing a plan based on community involvement are important elements 
in defining a vision and establishing a climate that is conducive to learning. By asking the right 
questions, such as those that have been presented here, school boards can keep their districts 
constantly focused on the student achievement agenda. 
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Cherry Creek: Public Engagement in Support of 
Academic Excellence 



The Cherry Creek (Colorado) School District found a way to engage the entire communi- 
ty in its quest to maintain academic excellence in the face of rapid growth and substantial 
budget cuts. Through an extensive process that took several years, the district built com- 
munity support for its mission of inspiring every student "to think, to learn, to achieve, to 
care. 

In 1993, the school board and administration convened an Education Summit to define 
and build a consensus for its vision of schooling. Faced with more than $34 million in 
budget cuts over a five-year period, the school district brought together 500 educators, cit- 
izens, parents, students, business leaders, and politicians from the Metropolitan Denver 
area to devise a funding system based on academic priorities. At issue: What really mat- 
ters in public education? 

Meeting in small groups over several weeks, summit participants debated what programs 
were of chief importance. During the next year, hundreds more people joined in the sum- 
mit process, which ultimately produced a series of recommendations. After debating the 
summit's recommendations for several months, the school board created a "tier" system 
of priorities, assigning priority to programs based on the extent to which they supported 
the district's principal goals. 

As it worked on balancing its 1995-1996 budget, the board kept in mind the communi- 
ty's desire to support academic programs and to reduce or eliminate programs in the lower 
tiers. It made cuts in transportation and some support areas and, rather than eliminate 
athletics, it established fees to generate additional revenues for sports programs. 

In fall 1996, the school board wanted to do still more to maintain the high level of edu- 
cational achievement in this district — the third largest in Colorado — as it grew by more 
than 1,000 students a year and served an increasingly diverse and needy student body. 
Expanding on the work of the Education Summit, the district asked a task force — a diver- 
sified group of board members, parents, teachers, administrators, and business leaders — 
to engage "in search of excellence." The task force visited and gathered information from 
several large school districts around the country rated among the top 10 nationwide for 
academic performance 

The task force then presented its findings at a series of community forums and shared 
them with staff at each school. Several months later, it presented its preliminary recom- 
mendations, shaped by community responses, to the school board. The district then used 
a series of electronic forums and an automated telephone survey to further assess the com- 
munity's perceptions of and reactions to the recommendations. 

Finally, the school board unanimously adopted a final set of recommendations aimed at 
increasing achievement for all students. The recommendations centered on decreasing 
class size, increasing the number of students who take the district's core academic subjects, 
providing more Advanced Placement courses, reserving additional teaching slots for high- 
poverty schools, and taking other measures to support high expectations and achievement. 
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Part Three: 

A Guide To Testing 




S chool is concerned with a number of outcomes, of which student achievement is only one, 
albeit a crucial one. Schools must also concern themselves with preparing students to par- 
ticipate in the civic life of a democracy and with inculcating habits of lifelong learning. Still, 
for many school districts, student achievement, as measured by standardized tests, represents 
the bottom line. Test scores can drive public support, affect the tenure of the school board and 
superintendent, influence private school enrollments, and, of course, affect day-to-day class- 
room programs and activities. 

Because tests play such a significant policy role in the decisions that local school boards make, 
they (and the public) must understand what each test does, how well it does it, and what test 
scores reveal about student achievement. 



Measuring Academic Achievement: Test Scores 

Understanding what test scores reveal is not always a simple or straightforward matter. This sec- 
tion will help school boards understand what tests measure, the societal and familial contexts 
which can influence variations in test scores, how to compare student performance, issues con- 
cerning the statistical significance of test results, and the importance of focusing on education- 
al productivity. With this information, school boards will be better equipped to make sound 
judgments regarding the selection and use of tests in their particular school district. 



What Tests Measure 

With hundreds of school tests available, it is obvious that not every test measures the same con- 
tent or skills. If a test is to be used to evaluate what students have learned, it is important that 
there be a good match between what the test measures and the objectives of the school, pro- 
gram, reform, or innovations. 
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For instance, in the era following the 1957 launch of the Soviet satellite Sputnik, 
many mathematics textbooks presented math instruction through set theory. The 
tests given in the late 1960s, which contained questions based on that theory, 
used terms such as associativity, transitivity, null set, and empty set. If today's stu- 
dents took those tests, they would likely do very poorly. Yet, their poor per- 
formance would not reflect their knowledge of mathematics. Rather, it would 
reflect the fact the contemporary students have not learned the vocabulary asso- 
ciated with set theory and have not learned math in the way those particular tests 
expected them to learn. 



The point bears repeating: Before using a test to hold a program, school, or 
school system accountable, school boards must make certain that the content of 

the test is relevant to the goals of the school. They must ensure that the program, 

reform, or innovation could be expected to improve scores on that test — particularly if the test 
is an important element in a broader plan to assess student achievement. 



Nationally used commercial achievement tests are designed to be sold in large markets and so 
are not tailored to particular state curricula. Similarly, state tests may not reveal how well stu- 
dents learned local "add-ons" to the curriculum. Other factors, such as student motivation to 
prepare and work hard on the exam, may influence test results. Did students have a stake in 
doing well? 

While achievement tests are geared to specifics, other tests, such as the SAT I (hereafter simply 
called the SAT) attempt to measure more "global" cognitive development. For example, SAT 
tests may predict how well a student will do in the first year of college but do not adequately 
measure achievement on subject matter that is taught in high school. While some SAT ques- 
tions (about algebra and geometry, for example) clearly relate to instruction, other questions 
(e.g., analogies) do not. Very little if any instruction in school asks students to complete an 
analogy. However, because many consider analogical reasoning to be important to success in 
college, one part of the SAT verbal section includes questions of this type. Conversely, questions 
on history, geography, chemistry, and other subjects do not even appear on the exam. 



The Social/Family Context 



The familial and 
community factors 
in a child's life affect 
cognitive develop- 
ment at least as 
much as do schools 



While life in school is typically more focused on academics than life outside of school, children 
do not spend nearly as much of their lives in school buildings as elsewhere. Not 
suprisingly, the familial and community factors in a child's life affect cognitive 
development at least as much as do schools. In an analysis of data aggregated to 
the district level, one researcher found that out-of-school factors (i.e., parental 
education and other background variables) accounted for fully half of the total 
variation in mean student achievement. 
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In some states, however, students may perform better on national assessments 
than their family and demographic characteristics would predict. In Texas, for 

example, 44 percent of fourth -graders in disadvantaged urban schools performed 

at or above the "basic" level in the 1992 NAEP math assessment, compared with only 27 per- 
cent in California. Hispanic fourth-graders also fared better in Texas than in California — 45 
percent of Hispanic fourth-graders in Texas read at or above the "basic" level, compared with 
only 28 percent in California. These data should be viewed cautiously, however, because analy- 
ses of NAEP data below the state level involve high standard errors, making conclusions sug- 
gestive at best. 
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Observers say one factor in Texas' greater apparent success may be that state's investment in 
reducing class size in the early grades. Thus, out-of-school contextual factors should not be an 
excuse for low test scores. Board members should not find themselves saying, "We're doing 
well, given the kinds of students we have." They should, rather, take these contextual variables 
as a challenge and endeavor to implement strategies that successfully address the many factors 
that impact upon achievement. (Appendix B offers a more extensive discussion of social and 
family factors.) 



Comparing Student Performance 

In order to compare the quality or effectiveness of schools, programs, or educational reforms, 
studies must employ comparable samples of students. Test score comparisons are inappropri- 
ate when the samples of students taking the tests differ significantly. Unfortunately, this lack of 
comparability — which may result from sampling error, selection bias, or other causes — occurs 
often enough to illustrate with a few examples. 

Studies that compare how students in various countries perform on mathemat- 
ics and science tests are popular today, although, historically, these internation- 
al comparisons have been fraught with interpretative difficulties. For example, 
in the Second International Mathematics Study (SIMS) in 1987, SIMS officials 
refused to recognize Japanese trade schools as "schools" for the purposes of the 
study, even though they enrolled some 30 percent of all Japanese secondary stu- 
dents (none of whom were bound for Japanese colleges or universities). 

Omission of these potential test takers likely biased the sample strongly in 
Japan's favor. Similarly, at the twelfth grade, SIMS tested only those students still 
enrolled in mathematics courses. That decision meant that only two percent of 
Hong Kong's twelfth graders were tested, along with 15 percent of their 
American and 100 percent of their Hungarian counterparts. The fact that school 
enrollment figures differed dramatically across participating countries further 
complicated the study. Only 50 percent of age-eligible Hungarian students and 
17 percent of New Zealanders were even in school, compared with 87 percent of 
Americans. 

In contrast with earlier international comparisons, the Third International Mathematics and 
Science Study (TIMSS) is the most fair and accurate international comparison ever undertaken. 
Stria quality controls ensured that the sample of students taking the mathematics and science 
general knowledge assessments were representative of the entire population at the end of sec- 
ondary school. For example, in most countries with distinct education "streams" — such as aca- 
demic and vocational — students in all programs were represented in the TIMSS sample. Thus, 
the TIMSS represents a major improvement over previous international studies of student 
achievement by meticulously avoiding many of the pitfalls of sampling error and selection bias 
which charaaerized these earlier studies. (Note: Some seleaion difficulties remain at the 

twelfth-grade level for the TIMSS advanced mathematics tests.) 

Sampling error and seleaion bias also can influence certain state-by-state comparisons. Only a 
portion of the students in each of the 50 states take the SAT exam, and that percentage can vary 
widely from one state to another. Several state-level studies have examined the effea of partic- 
ipation rates on SAT scores. A recent study found that some 83 percent of the variability among 
states could be attributed to differential participation rates (Powell and Steelman, 1996). 
Similarly, an examination of SAT scores across time can be useful in looking at the performance 
of a school distria, but care must be taken to faaor in any demographic changes that might 
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affect scores. For example, if a school district increases its proportionate enrollment of at-risk 
students taking the SAT, a drop in scores may be more a reflection of socioeconomic factors 
than a decline in student achievement. While this is true, in setting achievement goals, socioe- 
conomic factors should not be seen as an excuse for low performance. 

Finally, rapid changes in demographics, which affect test scores independently of any change in 
the quality of schools often make trend comparisons difficult. In a school district like Fairfax 
County, Virginia, which has gone from being almost all white and English-speaking to being 34 
percent minority and having students who speak more than 100 languages, test score changes 
over time are difficult to interpret. As subsequent discussion will illustrate, testing programs 
that measure achievement for the same students at two or more points in time avoid this prob- 
lem (see "Assessing Educational Productivity," below). 



Statistical Significance 

Policymakers, administrators, and other stakeholders in the educational process often point to 
results that are "significant" without specifying what the word "significant" means. Does it refer 
to the results of a statistical test or imply important practical consequences? Results that are sta- 
tistically significant might or might not have practical implications. To understand this differ- 
ence, we need to know the meaning of "statistical significance." 






Statistical signifi- 
cance is . • . a 
statement of 
probability — it tells 
only how likely it is 
that the results 
occurred by chance. 
It does not reveal 
anything about 
the substantive 
significance of the 
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which, ultimately, 
is a matter of 
judgment. 



Researchers use tests of statistical significance to decide whether or not their results reflect like- 
ly differences, or whether they might well have happened by chance. For example, a reading 
test might be administered to two groups of students, one of which has received reading instruc- 
tion by phonics, the other using the whole language approach. Let us say the two groups have 
average scores that differ by some amount labeled as "D." A test of statistical significance 
explains how likely it is that one would find a difference as large as "D" if the two groups real- 
ly did not differ — as the statisticians say, if the two groups came from populations with the 
same mean. Statistical significance is, therefore, a statement of probability — it tells only how 
likely it is that the results occurred by chance. It does not reveal anything about the substantive 
significance of the observed differences, which, ultimately, is a matter of judgment. In terms of 
the foregoing hypothetical example, the difference in reading test scores between students 
receiving instruction by phonics and the whole language approach might be statistically signif- 
icant, yet too small to be of any substantive significance and, thus, of little interest to school 
— board members and other decision makers. Finally, it is important to note that 
tests of statistical significance are dependent upon sample size. The same "D" that 
is statistically significant in one study may not be statistically significant in anoth- 
er with fewer subjects. 



Understanding the concept of statistical significance is useful in interpreting test 
score results and in making comparisons among test takers. To cite just one exam- 
ple, the headline over one story which recently appeared in a major education 
journal read: "Good News: Our 9-Year-Olds Read Well; Bad News: Our 14-Year- 
Olds Don't." The good news was that in an international comparison of reading 
skills, American nine-year-olds had finished second among 27 nations. The 
"bad" news was that, among the 31 countries testing 14-year-olds, the United 
States had finished 8th. Idowever, a statistical analysis conducted by the National 
Center for Education Statistics at the U. S. Department of Education found that, 
of the seven countries ahead of the United States on the 14-year-old test, only one 
was ahead by a statistically significant margin. At the same time, because results 
from the top 16 countries were very close, one had to go all the way from the 8 th- 
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ranked U.S. to the 1 6th-ranked nation to find a country performing significantly below the U.S. 
Even then, one may still pose the question: Are such differences in rank substantively significant? 



Assessing Educational Productivity 



Assessing the productivity of our nation's schools is a complicated matter, not 
widely understood by board members, school administrators, teachers, or the 
public at large. Yet, as the principal intermediaries between schools and the com- 
munity, school board members must come to terms with some of the funda- 
mental issues concerning productivity assessment. Just as no industry in the pri- 
vate sector could operate successfully without meaningful productivity data, 
those charged with responsibility for our nation's schools require reliable and 
valid data to guide their decision making. 

Ideally, the measurement of productivity in education should include: ( 1 ) meas- 
uring both inputs and outputs; (2) establishing linkages between specific inputs 
and outputs (i.e., holding constant other, competing inputs); and (3) evaluating 
only what a particular production unit (e.g., school, classroom, etc.) adds to the 
product — in other words, assessing what economists refer to as "'value added." 



Inputs and Outputs — Inputs to the educational process may include such 
factors as curriculum, instructional approach, and teacher quality, as well as 
a variety of social and family context variables beyond the control of schools. 

Outputs generally include various factors associated with the cognitive, emo- 

tional, and social development of children. In the present context, a single output — student 

achievement — is of most concern. 
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simply to be conver- 
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• Establishing Linkages — Establishing clear linkages between specific educational inputs 
and student achievement requires a basic understanding of certain key issues concerning 
testing and research design. Inputs such as teacher quality or the number of computers in 
a school are typically confounded with other "non-programmatic" inputs, such as the social 
background of students. Thus, to make meaningful test score comparisons, one must sep- 
arate out the influence of those variables that are beyond the control of the school. One 
way to accomplish this is to collect information on family/social background at the time 
tests are administered and to use these data to statistically control for the effects of such 
extraneous variables when test results are analyzed. Another approach, which may be espe- 
cially useful in evaluating the efficacy of new curricula or other programmatic reforms, 
involves the use of random assignment to individual classrooms. Although difficult to 
implement, when students are randomly assigned to classrooms, the influence of family 
and social background variables is effectively removed. This approach was recently 
employed to provide the first clear evidence that smaller class size is linked to student 
achievement (Mosteller, 1995). 

• Assessing "Value Added" — In interpreting test scores, it is also important that board mem- 
bers and other stakeholders in the educational process begin to focus their attention on the 
"value added" by specific inputs. To evaluate value added, tests must capture how much 
achievement changes over time in response to specific interventions. By administering tests 
to the same students at two or more points in time, districts can assess net gains associated 
with the educational process. Unfortunately, local school districts have been slow to imple- 
ment such "before and after" testing programs, and those that have typically fail to use the 
resulting data to maximum advantage. 

J 1 




23 



School board members' understanding of these fundamental issues concerning the measure- 
ment of productivity in education is critical to the successful fulfillment of their mission. To 
make informed policy decisions, school board members of the 21st century must become 
increasingly sophisticated consumers of testing and evaluation data. It is not enough simply to 
be conversant with the conclusions of the latest evaluation or accountability study — board 
members also must begin to learn how to evaluate studies for their underlying methodological 
rigor and to weigh research conclusions based on that new knowledge. 



Types of Tests 

Having given some general considerations to tests and their interpretations, this next section 
describes various kinds of tests which school districts commonly use. They include: 

• Norm-referenced tests; 

• Criterion-referenced tests; 

• Performance tests; and 

• International tests. 



Norm-referenced Tests 



With the exception 
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With the exception of those tests constructed by teachers themselves, commer- 
cially developed, norm-referenced achievement tests are the most common form 
of educational achievement tests used in schools. The tests most commonly 
used include: the Iowa Tests of Basic Skills, the Stanford Achievement Test, the 
Metropolitan Achievement Test, the Comprehensive Tests of Basic Skills, and the 
California Achievement Test. 



Norm-referenced tests compare individual student performance with the per- 
formance of others (usually a national sample) in the same grade. To develop 
such a test, the test publisher asks experts in curriculum areas to review the most 
common materials and textbooks used in classrooms and to develop questions 
which reflect the content of those materials. These questions are then adminis- 
tered to a representative sample of as many as 200,000 students across the 
nation. The national norm of a test is the score that falls at the middle — or 50th 
percentile — of the entire range of scores in the norming sample. Half of all 
scores are below this norm and half are above it. It is worth emphasizing that this method of 
defining the norm ensures that half of all students are always "below average" — for some, this 
represents a disquieting consequence of norm-referenced testing. 




Norm -referenced test scores also can be reported in terms of "grade level." In this instance, 
grade level is defined as the score at the 50th percentile for a particular grade. Again, by defi- 
nition, half of the nation's students are always "below grade level." This method sometimes 
leads to confusion and occasionally misleading stories in the media. For instance, a newspaper 
might publish a story cautioning the public that 25 percent of students promoted from the 
eighth to the ninth grade were below grade level. Of course, even a relatively high-performing 
district would expect to have a substantial percentage of its students performing "below" grade 
level, when it is defined in this manner. Grade level, as used by test producers, says little or 
nothing about a student's readiness for the work at the next grade. 
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For many years, it was impossible to track trends, up or down, on most commercially developed 
norm-referenced achievement tests. After a test had been on the market a few years, its pub- 
lishers would revise and "renorm " the test, since curriculum changes and growing familiarity 
with the test might cause changes in performance not reflective of changes in actual achieve- 
ment. The publishers would administer the test to another national norming sample, and 
whatever raw score corresponded to the 50th percentile would become the new "national 
norm." This score might or might not be the same score established by the administration of 
the earlier version of the test. (See the more detailed discussion concerning renorming the SAT, 
below.) Thus, the "norm" in a norm-referenced test is often a floating standard — it is subject to 
change each time the test is renormed. As a result, renorming can have the effect of obscuring 
gains or losses in achievement. 

The Iowa Tests of Basic Skills (grades 3-8) and the Iowa Tests of Educational Development 
(grades 9-12) represent two important exceptions to this rule. By long-standing state law, each 
new form of these tests must be equated, statistically, to the previous form. It is, therefore, pos- 
sible to compare scores on the Iowas (as they are jointly called) from 1955 to the present. 
These trends show scores rising from 1955 to about 1965, falling from 1965 to about 1975, 
then reversing direction, attaining all-time highs for most grades around 1988. 

These year-by-year results are available only for students enrolled in schools in the state of Iowa. 
However, the results of national norming samples, conducted about every five years, show that 
national trends mirror the Iowa results very closely, reaching all-time highs in the late 1980s 
and remaining flat since that time. For the last two years, scores in Iowa have declined some- 
what, but no national norming sample is yet available to determine if a similar decline has 
occurred across the nation. 



Criterion-referenced Tests 

Criterion-referenced tests measure a person's performance against a well-specified set of stan- 
dards. Such specification is easy to accomplish in, say, ice skating, but very difficult in relation 
to curricular domains which often are only vaguely specified. For that reason, 
the enthusiasm that greeted the concept of criterion-referenced testing seems to 
have abated somewhat. Nonetheless, people still refer to some tests as "criteri- 
on-referenced." The term is typically used to refer to a test which comes with a 
"cut score" — a score students must attain in order to pass the test. 

The "minimum competency" tests employed by some 35 states during the 1970s 
were of this type. Although these tests were usually referred to as criterion-refer- 
enced tests, the "criterion" was simply getting some percentage of the items cor- 
rect. If the cut score was set at 70 and the test was 100 items long, a student could 
pass by missing the first 29 questions and getting the next 71 correct, or by 
answering the first 71 correctly and missing the last 29. A more refined approach 
to criterion-referenced testing would take these differences into account. 

Criterion-referenced tests have one important advantage over norm- referenced tests — they can 
more readily be developed to evaluate programs, reforms, and innovations. Norm -referenced 
tests come off the shelf and, generally, cannot be tailored to specific programs. Criterion-refer- 
enced tests are more likely to be developed as assessment tools for locally-developed programs 
and innovations. (In terms of content definition, the NAEP — described later in Part Three — 
represents the only criterion-referenced assessment employed on the national level.) 
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Performance Tests 



Performance tests require students to demonstrate directly what they can do, through writing 
essays, performing mathematical computations, conducting scientific experiments, or provid- 
ing portfolios of artistic work. These tests are attempts to provide alternatives to the more 
common standardized tests. While multiple-choice tests can evaluate a large number of peo- 
ple quickly and inexpensively, they are generally less effective than performance 
tests in evaluating curriculum areas such as writing. Furthermore, in addition to 
their role in assessing achievement, performance tests can be useful as devices 
from which students leam even as they are being tested. 

Performance tests are likely to find a niche where they can be used in instruc- 
tional settings. More than the other types of tests discussed, they offer an oppor- 
tunity to directly assess competence. Performance tests are frequently used to 
train airplane pilots, for example. While the public is not necessarily interested in how well a 
future pilot performs on a multiple-choice test of navigation, it is considerably interested in that 
pilot's ability to get the airplane up in the air, to the right destination, and down again safely. 
A performance test, using simulators, assesses those abilities better than a multiple-choice test 
would. 

Performance tests have been less useful as accountability instruments, in part because of the dif- 
ficulty in obtaining reliable scores, but also because they are costly to implement and consume 
a good deal of time if given to all students. In one instance where an entire state has adopted 
a performance assessment, the technology has not yet proven wholly adequate. Vermont adopt- 
ed the use of portfolios for assessment and, thus far, has had trouble obtaining reliable results 
when the portfolios are scored by several scorers. In an instructional setting, where several 
teachers might share with a student their opinions of his or her writing, differing views of qual- 
ity need not be a problem. For evaluation purposes, however, those varied opinions can make 
objective assessments about quality difficult to attain — one teacher may evaluate a student's 
work as excellent, whereas another might conclude that it is adequate, at best. 




International Tests 

International tests are not a "type" of test but are treated separately because, in some quarters, 
they have garnered prominence. Many American educators, together with the public and the 
news media, want to know how well this nation's students are performing in comparison with 
those of other countries. Each international comparison has become more sophisticated in its 
methodology, although interpretive problems remain: Can there be truly equivalent test items 
in so many different languages? Are questions free of cultural influence? 

According to results from the Third International Mathematics and Science Study (TIMSS), U.S. 
students performed relatively well in these subjects at the fourth grade level — above the inter- 
national average in both math and science and outperformed in science by only one country 
(see box). U.S. students' relative standing in math and science declined, however, by eighth 
grade and continued to decline relative to other TIMSS countries in the secondary school years. 
U.S. twelfth graders outperformed terminal-year secondary school students in only two coun- 
tries — Cyprus and South Africa. 



International Comparisons in Mathematics, Science, and Reading 
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] TIMSS, 1996 & 1998 

OEA International Reading Literacy Study, 1992 



The results of an international comparison of reading literacy, conducted by the International 
Association for the Evaluation of Educational Achievement (IEA) present a somewhat brighter 
picture. Overall, U.S. fourth- and eighth-grade students were outperformed by only one coun- 
try (Finland) and, for both grades, both ages in the reading study, the 90th, 95th, and 99th per- 
centiles of American students scored the highest in the world. That is, the best readers in the 
United States scored higher than the best readers in all other nations, including Finland. 



The NAEP and the SAT 



Two tests — the National Assessment of Educational Progress (NAEP) and the Scholastic 
Assessment Test (SAT) — have become nationally prominent in the last 20 years. Because these 
two tests are often used by the media as indicators of school quality, they are discussed in some 
depth below. 



National Assessment of Educational Progress (NAEP) 

About 35 years ago, then-Commissioner of Education Francis Keppel gave his support to an 
idea advanced by Ralph Tyler to survey the nation's students to determine what they knew and 
didn't know, much as health agencies surveyed the nation to gauge the prevalence of health 
problems. Keppel's proposal caused a firestorm of protest much greater than — but very similar 
in kind to — objections raised more recently to President Clinton's proposal for a national test. 
Local control of education is endorsed by many today, but it was even more hallowed then. 
People argued that a national assessment would stifle local innovations and creativity. More 
important, many in Congress and virtually all educational organizations protested that such a 
project would lead inexorably to a national curriculum and federal control of education. 



As a compromise, NAEP was originally housed in a non-federal organization for state policy, 
the Education Commission of the States, and forbidden to report any data at a level smaller 
than "region." It also reported data by gender and ethnicity. In 1 988, legislation was passed to 
permit the reporting of state-level data. Since then, several state-by-state comparisons have 
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Because too few 
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single district take 
the NAEP, it does 
not provide a 
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of achievement 
below the state level. 



been conducted in reading, mathematics, and science, although only about 40 
states participate at the state level. NAEP tests a variety of subject areas, with 
reading and mathematics assessments occurring every two years and science, 
writing, history, and geography tests occurring less frequently. 

Originally, NAEP was designed to reveal what most people know, what some 
people know, and what few people know. The testing companies were directed 
to construct some test items that 90 percent of the students would get right, 
some that 50 percent would get right, and some that only 10 percent would be 
expected to answer correctly. 



Later, a National Assessment Governing Board (NAGB) was organized, whose members want- 
ed NAEP to prescribe what people ought to know, not simply what they do know. To this end, 
NAGB constructed "levels" of performance: basic, proficient, and advanced. Many in education 
believed that the first attempt at setting such levels resulted in impossibly high standards, and 
many continue to believe that later levels are still unrealistically high. Moreover, several tech- 
nical studies of the process for setting the NAEP achievement levels found the process flawed. 



The nature of the NAEP levels will likely be a continued source of controversy, in part because 
they present a different picture of the accomplishment of students than some other studies 
show. For instance, the 1996 NAEP math report shows that only 18 percent of fourth graders 
were proficient, only two percent were advanced, and 32 percent were below basic. Yet 
American fourth graders performed well above the average score of the 26 countries participat- 
ing in TIMSS at the fourth-grade level. 



Trend results for the NAEP show a mixed picture. Science scores fell from 1969 to 1982 for 17- 
year-olds but have recovered most of the loss. Science scores for 9- and 13-year-olds were sta- 
ble until about 1994, when they began to rise. In mathematics, scores were stable from NAEP's 
inception until the late 1980s, but have risen now on three consecutive assessments. In read- 
ing, scores had inched up since NAEP's inception until the most recent assessment, when the 
scores of 17-year-olds fell slightly, while those of the other two age groups were stable. 



Because too few students in any single district take the NAEP, it does not provide a reliable 
measure of achievement below the state level. However, while some experts in testing caution 
against such use, the NAEP has been employed as a means of benchmarking district-level results 
on other assessments. For example, in interpreting student performance on a state-adminis- 
tered exam, district officials might weigh those results against the state's overall ranking on the 
NAEP — i.e., high district- level scores on a state-administered exam might be viewed more cau- 
tiously if the state's overall performance on the NAEP is low relative to that of other states. 
Recently, President Clinton has advocated a voluntary national examination that would yield 
reports at the district, school, and individual level — something that the NAEP, as it is now struc- 
tured, cannot do. 
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The Scholastic Assessment Test (SAT) 

In 1900, a group of colleges formed the College Entrance Examination Board to try to bring 
some measure of coherence to what was taught in the nation's high schools. Board members 
had noticed that similar-looking transcripts often concealed the fact that students had been 
exposed to very different course content. The board members thought that by administering 
examinations, they could demonstrate to high schools what they valued and, thus, encourage 
them to align their curriculum accordingly. 
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In 1901, the board tested 973 students in nine subjects. The next year it added four subjects 
and the number of students tested rose to 1,362. Impressed with the military's successes using 
more genera], multiple-choice ability tests in World War I to predict who would be good offi- 
cers, tank drivers, etc., the board decided to abandon the rather cumbersome process of con- 
structing many subject-area tests. In 1926, it opted in favor of developing a single test of 
"scholastic aptitude" that would predict who would be successful in college. The resulting test 
was called the Scholastic Aptitude Test, a name that endured until 1994. Since the inception of 
the SAT, the test's developer has considered it a "mere supplement" to the rest of a student's 
high school record and has consistently warned against relying too much on test scores in mak- 
ing college admissions decisions— advice that was largely followed through the late 1970's. 

The SAT gained public prominence when the national average SAT score fell for 17 consecutive 
years from 1963 to 1980. While the committee which the College Board appointed in 1977 to 
study the decline pronounced the fall a complex matter, the media and the public took it sim- 
ply as an index of school quality. Each change in SAT scores has been carefully scrutinized ever 
since. Even though fewer than half of the nation's seniors take the SAT, and virtually all of those 
are college bound, many people inappropriately view the test as a measure of the health of the 
nation's schools. 

Today, about 1.8 million students take the SAT each year. It is not an achievement test in the 
usual sense — that is, it is not linked to any particular curriculum — and some of its questions 
(analogies, for example) call for skills not specifically taught in schools. Thus, for many years, 
the College Board and the ETS maintained that the SAT was completely impervious to coach- 
ing, a claim which facts no longer support (although both institutions still maintain the impact 
of coaching is small). 

The SAT has evolved in several ways since its beginnings. In 1994, the College 
Board changed the content of the test to bring it more in line with "achievement" 
and renamed it the Scholastic Assessment Test. Since then, questions have been 
more closely aligned with high school experience. The math section, for exam- 
ple, replaces five questions in each of its two sections with 10 questions that ask 
the student to produce a response rather than select one of five multiple choices. 

In 1996, the College Board also renormed or "recentered" the SAT to make it 
more applicable to today's student population. Scores were last centered in 
1941, on the eve of World War II. The standard setters of 1941 were 10,654 high 
school students who predominately resided in the Northeast. Most planned to 
attend either Ivy League or Seven Sisters colleges or other private institutions. 

They were 98 percent white and 61 percent male, and fully 41 percent had attend- 
ed private, college-preparatory high schools. That group's average raw score was 
assigned the scaled score value of 500 — with a scale ranging from 200 to 800 points. 

The College Board recognized that the original group members were an elite who did not 
resemble today's million-plus test takers, who are 29 percent minority, 52 percent female, and 
83 percent public school attendees. In 1996, the board reasoned that if the score of 500 is to 
accurately represent the average score of students applying to college, the scale had to be 
changed. Prior to renorming, students scoring 450 might come away falsely believing that they 
were below average among students currently applying to college. The board called this change 
a "recentering." For school board members interested in making comparisons over time, the 
Educational Testing Service (ETS), which constructs the SAT for the College Board, will provide 
a conversion table that permits all scores to be converted into the old scale or the recentered 
one. In addition to changes over time in the population taking the SAT, selection bias also 



Even though fewer 
than half of the 
nation's seniors take 
the SAT, and virtually 
all of those are col- 
lege bound, many 
people inappropri- 
ately view the test as 
a measure of the 
health of the 
nation's schools. 




makes current year state-to-state comparisons invalid. For example, some of the nation's low- 
est-achieving states perform quite well on the SAT. However, in many of these states, the 
American College Testing (ACT) program serves as the principal college entrance examination. 
In these states, only the best students — as few as 5 percent of seniors — take the SAT, compared 
with other states where the test is administered to 70 percent or more of seniors. 



The SAT does not predict a student's success in college uniformly well. In highly 
selective colleges, it works better than high school grades. In these colleges, many 
applicants will have the same grade-point-average — straight As. Not all of those 
with straight As will get the same score on the SAT, so the SAT allows more dif- 
ferentiation. Differentiation is, in turn, critical for prediction. If every student has 
an A average, no differential predictions can be made. For most other colleges, 
however, the SAT predicts success in college no better than high school grades and 



adds very little to the predictions made from grades and rank in class. 



Most of the foregoing discussion concerning the SAT also applies to the American College 
Testing (ACT) program, except that ACT questions are more oriented toward achievement. 
Those who developed the ACT were more interested in using the test for placement and guid- 
ance purposes than were those who developed the SAT. 



Final Notes on Testing 

Different tests are constructed for different purposes. Some tests are tightly linked to curricula, 
while others are not. Local school board members should ensure that the tests which their dis- 
tricts choose are either used for the purpose designated by the test developers, or that they pro- 
vide a sound rationale for any new purpose. 

Further, school boards may need to make contextual decisions about the way results will be 
judged and compared. In short, local school board members need to be familiar with the var- 
ious types of tests to determine if they can be validly used to evaluate curricula or programs. 
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Part Four 



Other Indicators of Achievement 
and the Quality of Schools 

A s noted at the outset of Part Three, tests are not the sole indicator of how a student, school, 
or system is performing. There is much more to education than percentile ranks. Thus, Part 
Four will provide local school board members with a description of indicators other than tests 
that are or can be used to indicate student achievement and the quality of schools. Some of 
these indicators are based on objective data, while others are more subjective in nature. Five 
broad categories of indicators are presented for board members' consideration: student-based 
indicators; teacher-based indicators; parent and community-based indicators; curriculum- and 
equipment-based indicators; and other indicators (see box). 



Other Indicators of Achievement 

and the Quality of Schools 

• Student-Based Indicators 

— Percentage of Students Taking College Admission Tests 

— Percentage of Students Taking Advanced Placement Courses/Tests 

— Percentage of Students Taking Algebra in the Eighth Grade 

— Percentage of Students Going on to College or Other Post-Graduation 
Alternatives 

— Percentage of Students Meeting State Mandates 

— Dropout and School Completion Rates 

— Promotion and Retention Rates 



• Teacher-Based Indicators 

— Percentage of Teachers with Majors or Minors in Teaching Areas 

— Percentage of Teachers Attaining Regular Certification 

— Teacher Expertise and Experience 

• Parent- and Community-Based Indicators 

— Parental Involvement 

— Level of "Consumer" Satisfaction With a District's Graduates 



• Curriculum- and Equipment-Based Indicators 

— Average Age of Textbooks 

— Adequacy of Scientific Laboratory and Field Equipment 

— Effective Use of Technology in Student Learning 

• Other Indicators 

— Programs to Develop the Whole Child 

— Engagement of School with External Organizations 

— Innovative Projects 

— Class Size 

— Per-pupil Expenditures 
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Student-Based Indicators 



Percentage of Students Taking College Admissions Tests 

The percentage of students taking college admissions tests reflects, in part, the academic 
emphasis of the district. However, this indicator should be approached gingerly. The Forgotten 
Half, a document written several years ago, pointed out that only half of Americas high school 
graduates went on to higher education. Schools, this report claimed, were orient- 
ed towards these students, forgetting the 50 percent who planned to enter the job 
market, seek some technical or vocational training, or enlist in the military. 

School boards certainly do not want to ignore the needs of any students; nor do 
they want to limit students' options. Increasingly, school districts are offering 
school-to-career programs with expectations and performance standards that 
allow students a range of choices after graduation. Because these "certificate" pro- 
grams link vocational training with a rigorous academic program, they enable students to get 
good jobs directly out of high school, join the military, or choose to attend a university or two- 
year community college. 




Percentage of Students Taking Advanced Placement Courses/Tests 

The College Board develops Advanced Placement (AP) courses and their accompanying tests. If 
a district does not provide College Board-specified AP courses, it can usually designate other 
"advanced" or "accelerated" courses to serve the same function. 

AP courses are offered in roughly half of American high schools. More than 900,000 students 
took one or more tests in 1997 (a nearly ten-fold increase over the 98,000 students who took 
them 20 years earlier). The tests are offered in 29 subjects, with U.S. History, English Literature, 
and Calculus being the most popular; more than 100,000 students take tests in each of those 
subjects. 

Again, taking demographic changes into account, changes over time in the proportions of stu- 
dents taking the various kinds of advanced courses can be revealing. The proportion of students 
taking AP courses and tests can provide some index of the degree to which a district is encour- 
aging students to (a) take challenging courses and (b) continue on to higher education. The 
test scores are one index of how well the district is preparing the students that it is challenging 
with this advanced curriculum. 



Percentage of Students Taking Algebra in the Eighth Grade 

Some years ago, the College Board said that high school algebra appeared to be a "gatekeeper" 
of academic achievement. Students who took algebra in the eighth or ninth grade were found 
to take a much stronger academic program in later years and to be much more likely to attend 
college. The board recommended that more students should be encouraged to take algebra, 
and its Equity 2000 project works with a number of school districts to help ensure that all stu- 
dents complete Algebra I by the end of ninth grade. 
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However, while there is a correlation between taking algebra early and taking 
more advanced math and science courses later on, that correlation might not 
mean that algebra is a gatekeeper. It might well be that students who take alge- 
bra in the eighth and ninth grades are those whom schools have identified as 
capable of handling an academic curriculum and have encouraged to take high- 
er-level courses. Schools should be careful to ensure that all students who have 
the ability to take algebra in the eighth or ninth grade have that opportunity. On 
the other hand, they should not pressure those who cannot handle algebra to take that sub- 
ject too early, because those students may become frustrated and discouraged. 

According to Education Watch, a state and national data book prepared by The Education 
Trust — a Washington, D.C.-based organization created to promote high achievement for all 
students — algebra placements may be less objective than the public believes. Its 1996 report 
said that researchers who reviewed the relationship between performance on the 
Comprehensive Test of Basic Skills and placement in algebra in one large California school dis- 
trict found some glaring inequities. They found that 100 percent of the Asian and 87.5 percent 
of the white students performing in the top quartile of the test were enrolled in algebra, while 
only 51 percent of the African-American and 42 percent of Latino top-quartile students were 
enrolled. The study found that Asian students who performed in the third quartile were more 
likely to be placed in algebra than African-Americans and Hispanics who scored in the top quar- 
tile. 




Percentage of Students Going on to College or Other Post- 
Graduation Alternatives 

Many schools and districts often simply report the proportion of students going on to college, 
without necessarily specifying whether those colleges are two- or four-year institutions. 

Nationally, over the last few decades, school districts have attempted to encourage more stu- 
dents to enter the college track. In 1994, a chart in Education Week showed that 62 percent of 
June's high school graduates were enrolled in college the following fall. That publication also 
reported, however, that studies indicate that at least 75 percent of young people want and expect 
to earn a college degree. The gap between college expectations and actual enrollments also may 
serve as a useful indicator, suggesting the extent to which our schools adequately prepare stu- 
dents for college entrance. 

In addition, schools might want to look at the proportion of students who are in the kind of 
post-high school program they find appropriate. How many are in college? How many are in 
the military? How many are in jobs that are not low-paying, dead-end positions? How many 
are in vocational training programs? How many are not enrolled in any post-school activity? 
A profile of participation in the various post-graduation alternatives would seem to be one 
appropriate way to determine if the school or system is a high-achieving one. 



Percentage of Students Meeting State Mandates 

The proportion of students taking the kind and number of courses required for advanced diplo- 
mas could constitute an index of achievement at the school or system level. After the 1983 pub- 
lication of "A Nation At Risk," a number of states created differentiated diplomas. Students 
could earn a standard diploma for taking a certain number of courses or receive an "honors 
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diploma" by taking more courses— usually four years of math and social studies, three years of 
science, and at least two of a foreign language. (Most states already required four years of 
English.) Care must be taken to maintain the integrity of courses offered. A study on this issue 
from the University of Wisconsin found no evidence that courses were being watered down as 
more students attempted them. 

State mandates also come, increasingly, in the form of tests. The sanctions applied from these 
tests vary from state to state — students can fail to obtain a diploma, principals can be trans- 
ferred, and schools can lose their accreditation or even be taken over by the state if they do not 
meet the standards established by the tests. Most of these mandates come with some means of 
punishing schools that fail to meet the mandate; few come with any way of celebrating those 
who exceed it. 

The percentage passing such tests can be used as an index of achievement as long as everyone 
is satisfied that the test reflects what is being taught in the classroom. One variation on "per- 
centage passing" overall is to report the proportion of students passing on the first attempt, on 
the second attempt, and so on. The goal would be to increase the number of students who pass 
the tests the first time. 



Dropout and School Completion Rates 

The dropout rate is often one of the most difficult statistics to report accurately. Different dis- 
tricts have different ways of defining "dropout." The definition sometimes includes students 
who have merely left one district for another a thousand miles away. For many districts, a stu- 
dent is not a dropout until a certain period of time (highly variable) has passed and no other 
district has claimed that student. 

Nationally, the U.S. Department of Education defines a dropout as a person aged 16-24 who is 
not enrolled in school and who is not a high school graduate. By that definition, the dropout 
rate declined from 14.1 in 1976 to 11.1 percent in 1996. Although dropout rates for all races 
declined in the last two decades, in 1966 they still remained nearly twice as high for blacks (13 
percent) and four times as high for Hispanics (29.4) as for whites (7.3 percent). School boards 
may want to take careful stock of the dropout prevention initiatives they have implemented or 
may want to consider initiating alternative education programs if they have not already done 
so. 




As an indicator, percentage of on-time graduation has some drawbacks. If pressure is brought 
to bear on a district to increase on-time rates, there may be some danger of increasing the num- 
ber of students graduated for "seat time." This indicator would be best used in conjunction 
with others that relate to the maintenance of standards and special programs to help lower 
achieving students to reach them. 



School completion rates focus on those students who receive a high school diplo- 
ma or its equivalent. According to the U.S. Department of Education, in 1950 — 
which school critics think of as the "good old days" — only 34 percent of the pop- 
ulation completed four years of high school. In contrast, school completion rates 
today show that about 86 percent of 22-year-olds have completed high school or 
its equivalent. 
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Promotion and Retention Rates 



In recent years, the issue of "social promotion," in apparent contrast to promo- 
tion for achievement, has been debated extensively. Most researchers generally 
oppose the idea of retention, arguing that retention in grade is not a particular- 
ly attractive alternative. According to a study by John Hattie at the University of 
North Carolina at Greensboro, retention in grade was the worst of 41 innova- 
tions introduced to improve achievement, and was one of the few that actually 
had negative results. A Johns Hopkins study of inner-city Baltimore students 
found, however, that those who repeated a grade in elementary school saw their 
grades, test scores, and self-esteem improve. 



One alternative for 
students who are at : 
risk of grade reten- 
tion (or simply 
achieving well below 
their peers), is to 
provide extra 
assistance before 
promoting. 



When children are retained, they seldom blossom the next year into high achievers. They do 
better as they repeat the material for a second year, but they often struggle. One alternative for 
students who are at risk of grade retention (or simply achieving well below their peers) is to 
provide extra assistance before promoting. The extra assistance might occur in school, after 
school, during summer school, or even during weekends. Some researchers contend that 
schools should be flexible enough to give all children the instructional depth and time they 
require to make progress. 



Teacher-Based Indicators 



Percentage of Teachers with Majors or Minors in Teaching Areas 

Teachers cannot teach what they do not know. While subject knowledge needs to be augment- 
ed by skills in pedagogy, teachers must first know the subject matter. Unfortunately, many 
teachers do not have even a minor in the subject they teach. At the secondary level, 23 percent 
of teachers do not have even a college minor in their main teaching field. The problem is exac- 
erbated in high-poverty schools — one-third of public school teachers in high-poverty areas are 
without a minor in their main field, compared with only eight percent of teachers in low-pover- 
ty schools. (For further discussion of this issue, see What Matters Most: Teaching for America's 
Future, a 1997 study from the National Commission on Teaching and Americas Future.) 

While possessing a major or minor in a subject area other than education is no guarantee of a 
high-quality teacher, teachers who have majored in a subject are likely to be more committed 
to it and to bring more enthusiasm to its teaching. Different considerations may exist at the ele- 
mentary school level, of course. There, teachers often need expertise in many subject areas but 
may not necessarily need a minor in mathematics, for example, to teach that subject at a third- 
grade level. 



Percentage of Teachers Attaining Regular Certification 



This indicator of achievement is currently controversial in some quarters. Some hold an "any- 
one can teach" belief, while others contend that training in pedagogy is absolutely crucial. Even 
within the ranks of those who believe that certification is necessary, many argue that the certi- 
fication process is badly flawed. Perhaps the most thoughtful report on this issue is What 
Matters Most: Teaching for America's Future, which critiques the teacher preparation and certifi- 
cation process: 
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Although no state will permit a person to fix plumbing, guard swimming pools, 
style hair, design a building, or practice medicine without completing training and 
passing an examination, more than 40 states allow districts to hire teachers on 
emergency licenses who have not met these basic requirements. States pay more 
attention to the qualifications of veterinarians treating the nation's cats and dogs 
than to those of teachers educating the nation's children and youth (p. 14). 

The National Commission on Teaching and America's Future, together with many other educa- 
tion organizations, support a "three-legged stool" approach to help ensure teacher quality: 
accreditation, licensing, and certification. The National Council for Accreditation of Teacher 
Education has developed standards for teacher education programs. Under the auspices of the 
Council of Chief State School Officers, a consortium of more than 30 states and professional 
organizations has created performance standards for initial teacher licensing. And the National 
Board for Professional Teaching Standards has established standards and assessments for certi- 
fying accomplished teachers. School board members may want to leam more about these 
organizations' standards, which could help guide local teacher hiring and evaluation policies. 



Teacher Expertise and Experience 

Although several studies have found no relationship between teacher education 
levels and student outcomes, a quantitative review of 46 studies — all of which 
employed data averaged over schools or districts — concluded that there is a pos- 
itive relationship between levels of teacher education and levels of student test 
scores (Greenwald, Hedges, and Laine 1996). Another study, published by the 
New York City Board of Education, which compared high-achieving and low- 
achieving schools that had similar demographics, found that differences in 
teacher qualifications accounted for a substantial portion of the variation in stu- 
dent achievement in reading and mathematics. Additional studies, conducted at 
the classroom level, will be required to determine the precise magnitude of the 
relationship between teacher expertise and student achievement. 



In many districts, the most inexperienced teachers often get the toughest assignments. Seniority 
rules frequently allow more experienced teachers a choice of assignments, and many elect to 
work with the most academically advanced students or in schools or classes with fewer disci- 
plinary and related problems. Board members might wish to assess the presence of this policy 
in their district and, if it exists, to assess its impact on teacher competency and morale as well 
as on student achievement. School boards may also want to review whether or not their recruit- 
ment policies are effective in attracting experienced teachers. 



Relatively few 
teachers have access 
to sustained, 
intensive profession- 
al development 
about their subject 
matter, teaching 
methods, or new 
technologies. . . 
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Professional Development Activities for Teachers 

The history of professional development in this country has often not been a particularly happy 
one. Nationally, relatively few teachers have access to sustained, intensive professional devel- 
opment about their subject matter, teaching methods, or new technologies, according to 
"Doing What Matters Most: Investing in Teaching Quality." Often, professional development 
activities are short-term and may not be what teachers want or need. This situation is exacer- 
bated in some schools where there is intense pressure to keep teachers in their classrooms dur- 
ing the school day. This pressure often forces staff development activities to take place in the 
evenings or weekends. 
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Many districts, however, are trying new approaches to professional development which connect 
teachers to each other through in-school teams and professional committees that include teach- 
ers from all schools in a district. These professional development opportunities, described in 
What Matters Most, share certain features. They are connected to teachers' work with their stu- 
dents, linked to concrete teaching tasks, organized around problem solving, informed by 
research, and sustained over time by ongoing conversations and coaching. The Key School in 
Indianapolis, for example, devotes every Wednesday afternoon to a faculty planning and evalu- 
ation meeting. While students listen to presentations from community members in the audi- 
torium, faculty members discuss a wide range of problems and make long- and short-term 
plans. 



Board members might want to explore the possibility of evaluation meetings in 
their districts and discuss with teachers ways to create more effective professional 
training opportunities. 



Parent- and Community-Based Indicators 



Parental Involvement 

Parental involvement has long been recognized as an important indicator of a schools success. 
In its 1997 publication, "National Standards for Parent/Family Involvement Programs," the 
National PTA notes that research findings document the "profound and comprehensive bene- 
fits for students, families, and schools, when parents and family members become participants 
in their children's education...." 




One — but certainly not the only — measure of involvement is the proportion of parents attend- 
ing PTA and other parent organization meetings. Unfortunately, this indicator is highly sensi- 
tive to demographics and outside realities. In some districts, for example, working parents who 
are extremely committed to their children's education may find it very difficult to attend PTA 
meetings at night, when they are exhausted from working long days and want to spend time 
with their children. Schools will need to be sensitive to the real-world demands on families 
when scheduling back-to-school nights, parent-teacher conferences, and other school-related 
events. 



Many school districts have found a variety of successful ways to involve parents in their chil- 
dren's education. The Niles (Illinois) Elementary School District, as just one example, has 
extensive family and community involvement programs which feature family education nights 
in math and other curricular areas, parent workshops on issues ranging from discipline to 
school readiness, and family field trips to local museums. The district's Family Resource Center 
lends parents grade-level "skill boxes," which contain activities that reinforce the school cur- 
riculum. Its school volunteer program allows parents and community members to make either 
a one-time or sustained commitment, and its regularly published newsletter keeps parents 
informed of school events. 
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Level of "Consumer" Satisfaction with a District's Graduates 



The word consumer is somewhat ambiguous. Education reformers who speak of consumer 
satisfaction typically have in mind the three major institutions which receive graduates: the 
workplace, the military and the institutions of higher education. However, parents also 
should be considered as consumers, as should the students themselves. 

One indicator of the level of consumer satisfaction would be to ask local busi- 
nesses their opinions of the academic preparation and workplace values of high 
school graduates. Feedback of this kind would have to be evaluated carefully, 
because some businesses might equate higher skill levels with specific job train- 
ing that is usually not offered, and might not be appropriate, in an educational 
setting. Some employers, too, may pay at or near the minimum wage but expect unrealistical- 
ly high skill levels for low wages. 




Another way to test consumer satisfaction is to survey parents and students, which many school 
districts do on an annual or biennial basis. Nationally, the annual Phi Delta Kappan/Gallup 
Poll of the Public's Attitudes toward Public Education also offers insights into the public's atti- 
tudes toward public schools. Last year, 46 percent of respondents gave their local public schools 
a grade of "A" or "B," compared with 56 percent of public school parents. School boards might 
want to employ a variety of formal and informal techniques to determine how parents, com- 
munity members, and businesses view their students' high school preparation. 



Curriculum- and Equipment-Based Indicators 

Average Age of Textbooks 

In some states, science textbooks still suggest that man might one day walk on the moon. The 
importance of current textbooks can vary by subject, however. Science and mathematics are in 
most critical need of current textbooks, since both content and approaches to the subject mat- 
ter have changed considerably in the last 25 years. In subjects such as English, where the canon 
of literature may not change very frequently, basic textbooks can perhaps be older but might be 
supplemented by contemporary literature. 

Along with the age of textbooks, achievement indicators may include whether or not texts are 
available in sufficient number for each child to have one, and whether or not the selection 
process is aimed at quality of content and relevance to school system standards and curriculum. 
(School board members may want to review the questions regarding the alignment of textbooks 
with high standards in Part Two.) 

Adequacy of Scientific Laboratory and Field Equipment 

The operative — and difficult — word in this area is "adequacy." Older equipment that is not 
state of the art might well be adequate for instruction in basic skills. In a few instances, older 
equipment might actually increase skills. In the days when computer memory was scarce, for 
example, programmers became much more adept at writing programs that conserved memory 
than they are today with the advent of multi-gigabyte hard drives. 



On the other hand, outdated computers cannot give students access to the 
Internet, which most districts consider an important research tool. Nor can they 
run sophisticated graphics software, such as "Dance of the Planets," an astrono- 
my teaching program which allows students to gain a better understanding of the 
universe by seeing the planets in their proper orbits around the sun and in their 
relationships to each other. 




Effective Use of Technology in Student Learning 

The mere physical presence of technology in school systems will not automatically raise stu- 
dent achievement. However, if technology resources are properly designed and used, as a part 
of the education program, student learning can improve. According to the CEO Forum on 
Education and Technology, there are four resource indicators of a well-designed learning envi- 
ronment: the availability of hardware (e.g., 2-5 students per computer); a high level of connec- 
tivity (e.g., Internet and high-speed lines); quality content (current digital content); and pro- 
fessional development (e.g., hours of training and years of experience). 




Additionally, the effectiveness of the use of technology can be measured by the extent that it is 
integrated and regularly used in the classroom. Used properly, technology is not simply skill 
training in the use of a machine and its software. Nor is it simply "running" a software program 
with no thought as to how it adds to an overall learning strategy. If used strategically, technol- 
ogy can aid in producing student-centered learning, improving higher-order thinking and learn- 
ing skills, providing universal access to information, fostering collaborative learning and team- 
work, and extending education communication outside the school. Indicators of success can 
include these latter factors as well. 



Other Indicators 



Programs to Develop the Whole Child 

As Part One of this primer indicates, education has numerous purposes beyond preparation for 
the work force. The measure of a system that places value on the development of other student 
attributes can be found in the quality and scope of its programs in art, music, character educa- 
tion, athletics, community service, etc. 



Engagement of School with External Organizations 

A host of activities can be envisioned under this indicator of achievement. Most obvious, per- 
haps, are partnerships with business and industry in school-to-work programs, but other possi- 
bilities abound and will depend on local circumstances. School districts can tap a local uni- 
versity for intellectual resources, inviting professors in various science departments to describe 
what they do, for example. Museums are often excellent resources for schools to use, as are com- 
munity libraries. 
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Innovative Projects 



The use of projects such as fairs and demonstrations permits students a longer-term, more 
intense involvement with a subject area. Such projects can be a source of pride for the students 
and, on occasion, a revelation for teachers. At one Colorado high school that had demonstra- 
tions where students constructed applications of geometry, some teachers were surprised to see 
that "average" students had constructed some of the cleverest applications. Translating aca- 
demic knowledge into real-world applications can be a powerful force in raising student 
achievement. 

These kinds of projects can often be augmented and increased in value through the kinds of 
connections with other institutions described in the preceding indicator. 



Class size 

While the debate over class size has been an ongoing one, a review of the literature generally 
found positive effects of smaller class size (Greenwalk, Hedges & Laine, 1996). A 1997 study 
indicated that small class sizes produced better achievement and better emotional climates than 
larger ones (Wenglinsky, 1997). The key seems to be the role of the teacher, and whether that 
role does or does not change, as class size decreases. 

The definitive study of class size comes from Project STAR in Tennessee, where classes in the first 
three grades were reduced to 13-17 students (Mosteller, 1996). The students in small classes 
showed increased achievement; those effects have now been sustained through the ninth grade. 
What makes the Project STAR results so important is that students and teachers were randomly 
assigned within each school to regular classrooms (22-25 students), small classrooms (13-17 
students), or regular classrooms with a teachers aide. Such assignment means that it was vir- 
tually impossible for any sort of selective bias (e.g., socioeconomic status of students, teacher 
training) to operate in favor or one treatment or another. There was a small gain for classrooms 
with an aide, but a much larger one for small classrooms, an effect, as noted above, that has 
been sustained since the students left their small classes at the end of grade 3. 



Per-pupil Expenditures 




One often hears critics admonishing, "Don't throw money at the schools," because money 
doesn't matter. University of Rochester Professor Eric Hanushek contends, in a controversial 
1989 article in Educational Researcher, that "There is no systematic relationship between money 
and achievement." However, two more recent re-analyses of Hanushek's review of some 65 
studies demonstrate that Hanushek's own data reveal a correlation between spending and 
achievement. 



Other studies have examined state-level NAEP scores, state-level SAT scores, and 
district-level test scores in relation to educational spending. All find a relation- 
ship — more money is associated with higher achievement. For example, one 
study found that for every $1,000 difference in state spending on education, there 
was a 15-point difference in SAT scores (after SAT participation rates had been sta- 
tistically adjusted to make them comparable across states). 
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Finally, contrary to the claims of some critics, spending on education has not "skyrocketed" in 
recent years. Between 1967 and 1992, new real spending for schools increased by 60 percent, 
but much of that went to special education and targeted populations. When expenditures for 
special education students are excluded, real spending increased by approximately 35 percent 
over the 25-year period. A more recent study indicates that spending on instruction in schools 
in the 1990s has been nearly flat. 



The Importance of Alternative Indicators of 
Success 



Tests scores, especially from state and national tests, will always play a significant role in evalu- 
ating academic achievement. However, tests alone do not tell the whole story of how well a sys- 
tem is educating its students. A host of other indicators, from class size to teaching training, 
can demonstrate that programs are rich and relevant. Those additional indicators of achieve- 
ment can also build community support. Local school boards must decide which indicators 
best fit their vision of their school system. 
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Conclusion 




P reparing America's children for the challenges of an increasingly global and information- 
driven society is our nation's most important priority. This primer was designed to help 
school board members fulfill their leadership role in public education and to assist them in 
raising the bar for student achievement in local communities across the nation. 

Substantive information was provided on setting standards, using tests, identifying indicators 
of success, and defining approaches for aligning resources to meet student achievement goals. 
This primer also presented board members with ideas on how to use that information in the 
context of their school board governance role. But no single document can provide all the 
information or answer all the questions. Therefore, a directory of resources to link readers to 
other information sources was also provided. After reading this primer, and using it as an on- 
going reference, local school boards should have a strong foundation to effectively approach 
their student achievement mission. 

There has been no other time in our nation's history that local school board service has been 
more rewarding to those board members who have the tools to lead. In an effort to provide 
those tools, NSBA will be producing more reports on various aspects of student achievement. 
Together, we can make a difference in providing the best education possible to the children of 
this great country. 
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Appendix A: 

Information Sources For 
Local School Board Members 



Key Student Achievement Resources 

One publication cannot provide school board members with all the available information on 
student achievement. However, it can provide easy access to the leaders in the field. 

The following index will put school boards in touch with important data (e.g., NCES, the TIMSS 
survey NSF), indicators of student achievement (e.g. NEGP), studies (e.g. AAAS, AERA, TIMSS), 
innovative ideas (e.g. NAB, NAS), state-by-state comparisons and activities (e.g., CCSSO, NCES, 
NEGP), and information developed for and about the local school board role in student 
achievement (NSBA). 

Achieve 

1280 Massachusetts Avenue, Suite 410 
Cambridge, MA 02138 
617/496-6300 
617/496-6361 (fax) 
http://www.achieve.org 

American Academy for the Advancement of the Sciences 

1200 New York Avenue, NW 

Washington, DC 20005 

202/326-6400 

202/371-9849 (fax) 

http://www.aaas.org 

American Educational Research Association 

1230 17th Street, NW 

Washington, DC 20036-3078 

202/223-9485 

202/775-1824 (fax) 

http://aera.net 

Association for Supervision and Curriculum Development 

1250 North Pitt Street 

Alexandria, VA 22314-1453 

703/549-9110 

703/299-8631 (fax) 

http://www.ascd.org 
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Council for Basic Education 
1319 F Street, NW, Suite 900 
Washington, DC 20004-1152 
202/347-4171 
202/347-5047 (fax) 
http:/ /www. c-b-e. org 

Council of Chief State School Officers 
One Massachusetts Avenue, NW, Suite 700 
Washington, DC 20001-1431 
202/408-5505 
202/408-8072 (fax) 
http://www.ccsso.org 

Education Commission of the States 
707 17th Street, Suite 2700 
Denver, CO 80202-3427 
303/299-3600 
303/296-8332 (fax) 
http://www.ecs.org 

Mid-continent Regional Educational Laboratory 

2550 South Parker Road, Suite 500 

Aurora, CO 80014 

303/337-0990 

304/337-3005 (fax) 

http://www.mcrel.org 

National Alliance of Business 

1201 New York Avenue, NW, Suite 700 

Washington, DC 20005 

202/289-2888 

202/289-1303 

http://www.nab.com 

National Association of State Boards of Education 

1012 Cameron Street 

Alexandria, VA 22314 

703/684-4000 

703/836-2313 (fax) 

http://www.nasbe.org 

National Center for Education Statistics (including TIMSS) 

555 New Jersey Avenue, NW 

Washington, DC 20208-5574 

202/219-1828 

202/219-1736 (fax) 

http : //www. nces.ed.gov/timss 
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National Center for Research on Evaluation, 
Standards, and Student Testing 
University of California at Los Angeles 
301 GSE & IS 
Box 152206 

Los Angeles, CA 90095-1522 
310/206-1532 
310/825-3883 (fax) 
http://www.cresst96.cse.uda.edu 

National Center on Education and the Economy 

700 11th Street, NW, Suite 750 

Washington, DC 20001 

202/783-3668 

202/783-3672 (fax) 

http://www.ncee.org 

National Education Goals Panel 
1255 22nd Street, Suite 502 
Washington, DC 20037 
202/724-0015 
202/632-0957 (fax) 
http://www.negp.gov 

National Center for Improving Student Learning and 

Achievement in Mathematics and Science 

Wisconsin Center for Education Research 

University of Wisconsin - Madison 

1025 West Johnson Street 

Madison, WI 53706 

608/265-6240 

608/263-3406 (fax) 

http://www.wcer.wisc.edu/ NCISLA 

National School Boards Association 

1680 Duke Street 

Alexandria, VA 22314-3493 

703/838-6722 

703/683-7590 (fax) 

http://www.nsba.org 

National Science Foundation 
4201 Wilson Boulevard 
Arlington, VA 22230 
703/306-1234 
703/306-0090 (fax) 
http://www.nsf.gov 




New American Schools 

1000 Wilson Boulevard, Suite 2710 

Arlington, VA 22209-3901 

703/908-9500 

703/908-0622 (fax) 

http://www.naschools.org 

U.S. Department of Education 
600 Independence Avenue, SW 
Washington, DC 20202 
800/USA-LEARN 
202/401-2000 
202/401-0596 (fax) 
http://www.ed.gov 
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Appendix B 



Opportunity to Learn Indicators — The 
Context of Achievement 

F rom birth to age 18 , children spend nine percent of their lives in schools. While this nine 
percent is likely to be a more intense and focused time than that spent out of school, it still 
leaves a good deal of room for a child's life to be influenced by non-school factors. This sec- 
tion discusses the most important out-of-school factors that can influence student achievement 
(see box). 



Opportunity to Learn Indicators 


• 


Teenage Pregnancy Rates and Participation in Parenting Courses 


• 


Percentage of Female-headed Households 


• 


Poverty Rate 


• 


Student Mobility Rate 


• 


Paternal and Maternal Education Levels 


• 


Percentage of Students for Whom English is Not Their Native Language 


• 


Number of Violent Incidents Per Year 


• 


Number of Annual Police Visits/Disciplinary Actions 


• 


Demographic Trends 


• 


Physical Condition of Schools 



These factors are not excuses for not holding students to high expectations, but they should be 
considered in evaluating both individual student performance and that of the district. Some of 
these factors, which relate to both education and the quality of students' lives, may also be cru- 
cial to consider as school boards develop their achievement plans. School boards may want to 
involve city, county, and community organizations, as well as local businesses, in programs that 
both raise achievement and improve the quality of their students' lives. These programs might, 
themselves, become indicators of student achievement. 

In this primer, Opportunity to Learn (OTL) variables are defined as factors, beyond the school 
system's direct control, that have a bearing on achievement. During discussions of national 
standards and national testing, OTL variables were the subject of heated debate. Many consid- 
ered it unfair to hold all students to the same standard without taking into account the context 
of the situation. How would schools in poor urban and rural areas meet the same standards as 
those in affluent suburbs? 



Other educators and policymakers agreed that taking OTL variables into account was merely an 
excuse for continuing low achievement. Thus, when New Jersey statistically adjusted test scores 
by taking into account students' socioeconomic status, it demonstrated that, had its students 
been more affluent, they would have scored as high as those in wealthier states. This adjust- 
ment, according to critics, simply obscures real problems. A high-poverty state, like New Jersey, 
does not have the same demographics as other states, and it needs to raise achievement levels 
fr ^ r the students it does have. 
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Opportunity to Learn Indicators 



Teenage Pregnancy Rates and Participation in Parenting Courses 

Nationally, teen pregnancy rates have been falling slightly for the last few years for all groups 
except Hispanics. These rates are still much higher than they were 25 years ago, and the rate for 
blacks and Hispanics is about four times the rate for whites. 

Independent of any moral or economic considerations, teen pregnancy has implications for 
education. Research has shown that seven out of 10 adolescent mothers will drop out of high 
school. In addition, children of teenage mothers are far more likely to be physically abused, 
abandoned, or neglected than the offspring of older mothers. These negative outcomes are 
reduced in the teens who are enrolled in parenting classes which provide instruction in pre- 
natal and post-natal care. To enable adolescent mothers to stay in school, some school districts 
also provide on-site day care for their young children. 



Percentage of Female-headed Households 

Single-parent households, especially those headed by a single female parent, are often affected 
by severe economic problems. These households are often below the poverty line, which was, 
as of late 1997, just under $16,000 for a family of four. Although the research is not conclu- 
sive on the psychological effects of single-parent families on children, single-parentness is often 
associated with the stresses of poverty, which are discussed below. 



Poverty Rate 

In 1995, more than 14 million children — some 20.8 percent of all children — were classified as 
poor. While not an excuse for not learning, poverty is a condition that affects many aspects of 
life. Consider this description from the Ohio Children's Defense Fund: 

Poor children are more likely to have mothers who get late prenatal care and are 
more likely to be bom small. Medical studies show that poor children are three 
times more likely to have stunted growth, often as a result of inadequate diets. 
Low-income youngsters are more likely to drown, suffocate or be injured in a fire. 

Poor children are also about twice as likely to have physical or mental disabilities. 
Childhood death rates for poor youngsters are three times as high as those for other 
children, often the result of unsafe living conditions. Poor children are seven times 
more likely to be abused or neglected. 

The effect of poverty can be seen in schools across America. A recent ll.S. Department of 
Education study of the federal Title I program reports that high-poverty schools put "disadvan- 
taged students in double jeopardy." School poverty "depresses the scores of all students in 
schools where at least half the students are eligible for subsidized lunch and seriously depress- 
es the scores when over 75 percent of students live in low-income households," the report says. 

As part of their achievement plan, school boards confronted with high poverty might want to 
investigate those strategies that are successful in educating poor children. Research has shown 



that, although curricula and methodology vary considerably from district to district, successful 
high-poverty schools have certain similar characteristics: high standards; the constructive use of 
test scores; a willingness to take every necessary step to meet students' needs; strong parental 
involvement; and a vision that includes goals and a plan to reach them. The Center for Research 
on Education of Students Placed at Risk, located at the Johns Hopkins University in Baltimore, 
can offer school board members information about successful strategies. 



Student Mobility Rate 

Students who move around a good deal, even within a single district, are much more likely to 
eventually drop out than those who have a stable school environment. 

Student mobility in this country is exceptionally high: 20 percent of children change schools 
annually. In urban centers, that figure may reach 50 percent in some schools as the result of 
family and economic instability. Studies show that this mobility produces difficulties for the 
children and their teachers. A child moving into a new school, or returning to his former 
school in mid-year, might be ahead of or behind his peers, or might have learned a very differ- 
ent approach to, say, science or mathematics. Many school districts are confronting this chal- 
lenge by evaluating new students shortly after they arrive and offering interventions as soon as 
they appear necessary. 

High student mobility also may affect learning among stable student populations. Classroom 
disruptions, due to the frequent arrival of students who are not familiar with the curriculum 
and classroom practices, may result in less instruction time and lower achievement for all stu- 
dents. 



Paternal and Maternal Education Levels 

These are among the most powerful predictors of school achievement that schools cannot con- 
trol. Well-educated parents are likely to want even more education for their children. They can, 
as well, bring intellectual power to bear at home and perhaps financial power to provide edu- 
cational materials. Finally, they have experience with the educational system and can assist 
their children to "work it" in their favor. 

Accordingly, a school system may wish to pursue training opportunities to assist parents, espe- 
cially parents with limited education, to help teach their children— including pre-schoolers 
whose early language development is so essential for success in elementary school. 



Percentage of Students for Whom English is Not Their Native 
Language 

While knowing another language is a wonderful asset overall, students must have a good com- 
mand of English to succeed in schools. Even those who speak colloquial English are likely to 
have much more difficulty with the formalities of reading and writing. Over the past 10 years, 
there has been a 50 percent increase in the number of students for whom limited knowledge of 
English is a learning barrier. Of these 3.6 million English Language Learners (ELL), 50 percent 
live in urban centers. They are likely to contribute to the high incidence of poverty at their 
schools, since 75 percent of these students receive a free school lunch. Some new immigrant 
groups not only speak little or no English but are illiterate in their native languages as well. 
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The quality of programming a district provides for its ELL students — and perhaps for their par- 
ents — can indicate a system's commitment to raising achievement. 



Number of Violent Incidents Per Year 

Violence in schools has become an increasingly serious problem now. Although no one can say 
with certainty how often and in how many schools incidents occur, the U.S. Department of 
Health and Human Services' 1995 "Youth Risk Behavior Surveillance" report offers some dis- 
turbing statistics. According to this report, 20 percent of students nationwide had carried a 
weapon during the 30 days preceding the survey, nearly 39 percent had been in a physical fight, 
and 4.5 percent had missed at least one day of school because they had felt unsafe either at 
school or when traveling to and from school. 

In December, 1997, President Clinton ordered the Department of Education and the 
Department of Justice to annually compile a list of violent incidents in schools. Before those 
lists become available, school districts may want to develop their own categories for classifying 
violent incidents and determine how they will treat certain offenses. 



Number of Annual Police Visits/Disciplinaiy Actions 

Excluding visits made by officers as part of a drug awareness or other instructional program, 
occasions where officers are summoned to schools because of an incident or threat should be 
compiled and tracked. Likewise, the number and trends in specific kinds of disciplinary inci- 
dents and actions provide an indicator of the learning environment. 



Demographic Trends 

Immigration patterns and socioeconomic shifts are the demographic trends mostly likely to 
have an impact on schools. The impact can be either positive or negative, although immigra- 
tion is likely to put stress on a school. In the First International Adult Literacy Survey published 
in 1995, the United States had far and away the highest proportion of immigrants reading at 
the lowest level. 

The nation's secondary schools also are seeing increases in the percentages of students older 
than 18 who attend. According to the U.S. Department of Education, the percentage of seniors 
in U.S. public high schools who are 19, 20, or 21 years old nearly doubled in a decade, rising 
from four percent in 1984 to seven percent in 1994. Many of these students are in special edu- 
cation programs, others are immigrants who do not speak English or were delayed starting 
school in their native countries, while still others are students whose family, educational, or 
social problems have prevented them from graduating earlier. 

As part of their overall achievement plan, school districts will need to determine how to best 
educate these populations. As one example, some school districts have opened "newcomer" 
schools for recent immigrants. 
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Physical Condition of Schools 



Many of the nation's schools are physically in very poor shape. The U.S. General Accounting 
Office (GAO) estimated that it would cost $112 billion nationwide to address such problems 
such as fire-code violations, building maintenance, air-quality problems, roof and other repairs. 
The GAO also said that nearly half the nation's schools have inadequate electric wiring for com- 
puter and communications technology. 

In addition, many schools districts, faced with enrollment surges, are overcrowded. The U.S. 
Department of Education estimates that 6,600 more schools will be needed by 2006 to house 
new students, and additional schools may be necessary to replace aging buildings. 

The physical condition of schools certainly affects morale, and school boards will want to 
determine what effect it has on technology and other programs, class size, and other elements 
of education. Many school boards might want to include advocacy for bond issues and tax 
levies as part of their overall achievement plans. 




59 



53 



About NSBA 



NSBA's Mission Statement 

The mission of the National School Boards Association, working with and through all its 
Federation Members, is to foster excellence and equity in public education through school 
board leadership. 

NSBAs Shared Vision 

NSBA and its Federation Members represent 95,000 local school board members who are ded- 
icated to educating every child to his or her fullest potential. In 1996, Federation leaders came 
together to forge a plan for increasing student achievement through school board leadership. 
This effort coalesced into a strategic vision — a vision of the National School Boards Association 
as a powerful, united, energetic Federation ... as the premier advocate for public education 
... as an influential force for achieving equity and excellence in public education . . . and 
as a catalyst for aligning the power of the community on behalf of education. 

Underlying this shared vision are certain bedrock convictions: 

• belief that effective local school boards can enable all children to reach their potential 

• conviction that local governance of public education is a cornerstone of democracy 

• belief in the power of local school boards to convene the community around education 
issues 

• conviction that together, local school boards can influence education policy and gover- 
nance at the state and national levels 

• commitment to the principle that through collaboration comes impact 

• belief that the strength of local school board leadership arises from the board's capacity to 
represent the diversity of students and communities 

The National School Boards Association is the nationwide advocacy organization for public 
school governance. NSBA's mission is to foster excellence and equity in public elementary and 
secondary education in the United States through local school board leadership. NSBA 
achieves its mission by amplifying the influence of school boards across the country in all pub- 
lic forums relevant to federal and national education issues, by representing the school board 
perspective before federal government agencies and with national organizations that affect edu- 
cation, and by providing vital information and services to Federation Members and school 
boards throughout the nation. 

NSBA advocates local school boards as the ultimate expression of the unique American institu- 
tion of representative governance of public school districts. NSBA supports the capacity of each 
school board — acting on behalf of and in close concert with the people of its community — to 
envision the future of education in its community, to establish a structure and environment that 
allow all students to reach their maximum potential, to provide accountability for the people 
of its community on performance in the schools, and to serve as the key community advocate 
for children and youth and their public schools. 

Founded in 1940, NSBA is a not-for-profit federation of state associations of school boards 
across the United States and the school boards of the District of Columbia, Guam, Hawaii, and 
the U.S. Virgin Islands. NSBA represents the nation's 95,000 school board members. These 
board members govern 14,772 local school districts that serve more than 46.5 million public 
school students — approximately 90 percent of all elementary and secondary school students in 
the nation. Virtually all school board members are elected; the remainder are appointed by 
elected officials. 

NSBA policy is determined by a 150-member Delegate Assembly of local school board mem- 
bers from throughout the nation. 
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